Motor vehicle artificial intelligence expert system dangerous driving warning and control system and method

ABSTRACT

Specifically programmed, integrated motor vehicle dangerous driving warning and control system and methods comprising at least one specialized communication computer machine including electronic artificial intelligence expert system decision making capability further comprising one or more motor vehicle electronic sensors for monitoring the motor vehicle and for monitoring activities of the driver and/or passengers including activities related to the use of cellular telephones and/or other wireless communication devices and further comprising electronic communications transceiver assemblies for communications with external sensor networks for monitoring dangerous driving situations, weather conditions, roadway conditions, pedestrian congestion and motor vehicle traffic congestion conditions to derive warning and/or control signals for warning the driver of dangerous driving situations and/or for controlling the motor vehicle driver use of a cellular telephone and/or other wireless communication devices.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 17/024,535 filed onSep. 17, 2020, which is a continuation of Ser. No. 16/563,427 filed onSep. 6, 2019 and issued under U.S. Pat. No. 10,814,784 on Oct. 27, 2020,which is a continuation of U.S. Ser. No. 16/168,449 filed on Oct. 23,2018 and issued under U.S. Pat. No. 10,434,943 on Oct. 8, 2019, which isa continuation of U.S. Ser. No. 15/885,412 filed on Jan. 31, 2018 andissued under U.S. Pat. No. 10,137,834 on Nov. 27, 2018, which is acontinuation of U.S. Ser. No. 15/277,037 filed on Sep. 27, 2016, issuedunder U.S. Pat. No. 9,919,648 on Mar. 20, 2018, all of which are hereinincorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

Motor vehicle accidents result in tens of millions of people beinginjured, disabled or even dying throughout the world each year. It hasbeen estimated that motor vehicle accidents cause over 1.3 millionworldwide deaths each year or over 3000 deaths each day. In addition tothe human tragedy, road crashes cost over $500 billion globally peryear. In the United States alone over 37,000 people die on road crasheseach year and an additional 2.3 million are injured or disabled. Clearlythe seriousness of the situation requires diligent efforts to reducemotor vehicle accidents. Vehicle design, roadway engineering, police andlaw enforcement efforts and technological improvements in vehicle safetyand warning systems are all important areas. See, for example,Association for Safe International Road Travel, 2002-2016.http://asirt.org/initiatives/informing-road-users/road-safety-facts/road-crash-statistics.

This invention addresses this growing and serious problem through aninnovative combination of technologies and advanced informationgathering and processing methods in a comprehensive integrated motorvehicle danger warning and control system. Information from automotivesensors, passenger activity sensors, driver medical condition sensors,roadway condition sensors, pedestrian congestion and traffic measurementsensors are combined in an integrated system making use of artificialintelligence expert systems to derive motor vehicle warning and controlsignals with the goal of minimizing the occurrence of accidents whilemaintaining driver communication capabilities to report dangeroussituations requiring immediate assistance. In this invention, artificialintelligence expert systems technology simulates human reasoning toderive logical responses to dangerous driving situations by integratinglarge amounts of information into informative driver warnings and motorvehicle control signals.

A particular concern in motor vehicle safety is the increased use ofcellular telephone technology. Recent studies indicated that by the endof 2015 there were as many as seven billion cellular telephonesworldwide. This compares to an equal worldwide population of about sevenbillion people. While this universal adaptation of cellular telephonetechnology has led to many improvements in the quality of life tobillions of people, it has, at the same time, introduced several uniqueproblems with serious consequences resulting from improper use ofcellular telephones.

A particular area of concern involves the use of cellular telephones orother wireless devices while driving motor vehicles. For example, theNational Safety Council (NSC) estimates that, in the United States, 21percent of crashes or 1.2 million crashes in 2013 involved talking onhandheld and hands-free cell phones. The NSC also estimates anadditional 6 percent or more of crashes or a minimum of 341,000 ofcrashes in 2013 involved text messaging. Thus, the NSC estimates aminimum of 27% of crashes involve drivers talking and texting on cellphones. These alarming statistics led to the NSC's call for a ban oncell phone voice and texting use while driving. See National SafetyCouncil, “Annual Estimate of Cell Phone Crashes—2013,” © NCS report,2015, page 1.

Furthermore, recent studies have concluded that uses of cellulartelephones for voice communication or texting are only part of theproblem. The AAA Foundation for Traffic Study has published a reportstating that it has been estimated that driver inattention has accountedfor 25% of all police reported crashes. Other studies indicate that suchinattention was a factor in 78% of all crashes or near crashes making itthe single largest crash causation factor. See AAA “Measuring CognitiveDistraction in the Automobile,” AAA Foundation for Traffic Study, June,2013, page 4. The AAA study concluded that on a cognitive distractionscale, driver conversations with other passengers in a motor vehicle,the use of hand-held cellular telephones and the use of hands-freecellular telephones all give rise to about equal cognitive distraction.In all three of these cases cognitive distraction varied from 2.27 to2.45 times that of non-distracted, single task driving conditions. Theuse of speech-to-text technology increases cognitive distraction toabout three times that of non-distracted, single task drivingconditions. Interestingly, hands-free cellular telephone technologyoffers minimal cognitive distraction advantage over the use of hand-heldcellular telephones and rates only slightly better than having aconversation with other passengers in the vehicle. See above AAA study,page 28. See also, NSC, “Understanding the distracted brain—Why drivingwhile using hands-free cell phones is risky behavior.” National SafetyCouncil White Paper, April, 2012.

The seriousness of the situation has led to various technologicalsuggestions for reducing or eliminating the use of cellular telephonesby drivers while operating a motor vehicle while still allowing otherpassengers in that vehicle to use their cellular telephones for voicecalls and texting in a normal manner. Prior art systems and methodsattempting to address this need include the following:

Joel Vidal and Yael Vidal, U.S. Pat. No. 8,538,402, “Phone That PreventsTexting While Driving,” is directed in part to determination that a userof the phone is sitting in a driver seat of a moving vehicle based oncaptured images and/or contextual analysis of voice or text messages.Use is also made of GPS location information and possibly otherparameters to control cellular telephone usage.

Saled Tadayon and Maryam Halavi, U.S. Pat. No. 8,145,199, “ControllingMobile Device Functions,” assigned to BT Patent LLC, is directed in partto controlling mobile device functions by limiting or disabling featureswhich may cause distractions to the user including the use of cellulartelephones for voice or texting purposes. Use is made of a coded signaltransmission from a transmitter in the vehicle to assist in determiningthat the cellular telephone of concern in the moving vehicle is beingused by the driver of that vehicle.

Curtis A. Vock and Perry Youngs, U.S. patent application Ser. No.12/818,044 (Publication No. 20100323615—abandoned) and 20130288744, aredirected in part to a mobile device that includes a GPS sensor and amotion module that disables communication through the mobile device whenin motion. A limited range transmitter/receiver configuration is used toassist in the determination that the mobile device of concern is beingused by the driver of a moving vehicle.

Robert L. Mendenhall, et. al., “Intra-vehicular Mobile Device UsageDetection System and Method of Using the Same,” U.S. Pat. Nos. 8,295,890and 8,060,150 are both directed in part to the use of a directionalantenna in a vehicle configured and positioned to detect mobile deviceradio signals from the driver area in the vehicle with storage of mobiledevice usage data that may be useful to the trucking, train, bus, andmass transit industries in order to educate drivers about the dangersassociated with using mobile devices while driving.

Camp, Jr., et al., U.S. Pat. No. 7,697,917, “Method for safe operationof mobile phone in a car environment,” is directed in part to electronicequipment utilizing a wireless signal to communicate, includingdetermining if the electronic equipment is operated within a movingvehicle based on a characteristic of the wireless signal and inhibitingoperation of the electronic equipment if the electronic equipment is ina moving vehicle. Possible use of near field communication sensors (NFC)to determine if the driver is using the electronic device is discussed.

Abramson, et al., U.S. Pat. No. 8,750,853, “Sensor-based determinationof user role, location, and/or state of one or more in-vehicle mobiledevices and enforcement of usage thereof,” is directed in part toanalysis of one or more inputs originating at one or more sensors of oneor more devices in a in a moving vehicle to assist in determination ofvehicle class, the in-vehicle location, hand held state of a mobiledevice with systems and methods for restricting operation of a mobiledevice, including restrictions that impede operation by a driver more sothan operation by a passenger. Possible use of near field communicationsensors (NFC) to determine if the driver is using the electronic deviceis discussed.

Slusar, et al., U.S. Pat. No. 9,086,948, “Telematics based on handsetmovement within a moving vehicle,” is directed in part to a system forproviding telematics data associated with a vehicle being driven by adriver obtained by tracking the movements of a wireless communicationsdevice of a driver of the vehicle. The telematics data may provide,among other things, speed, acceleration, deceleration, times ofoperation, duration of operation, mileage driven per day, and day of theweek the vehicle has been used. Possible use of near field communicationsensors (NFC) to determine if the driver is using the wirelesscommunications device is mentioned.

Xiao, et al., U.S. Pat. No. 8,634,816, “Limiting mobile device servicesin an automobile,” is directed in part to a system for determiningwhether a mobile communication device is in a driver compartment of anautomobile. The method may include determining whether the automobile isin motion or not in motion. Further, the method may include redirectinga call to the mobile communication device when the mobile communicationdevice is in the driver compartment and the automobile is in motion.Possible use of near field communication sensors (NFC) to determine ifthe driver is using the wireless communications device is mentioned.

Additional prior art directed to technologies useful in some embodimentsof the present invention includes:

M. Brandstein and D. Ward, “Microphone Arrays,” Springer, Berlin,Germany and New York, 2001, dealing with the configuration andtheoretical foundations of microphone arrays.

J. Benesty, et. al., “Microphone Array Signal Processing,” Springer,Berlin, Germany and New York, 2008, dealing with the configuration andtheoretical foundations of microphone arrays and digital signalprocessing of signals produced from such arrays.

Chen, C. H., “Fuzzy Logic and Neural Network Handbook,” McGraw-Hill, NewYork, 1996.

Cox, C., “The Fuzzy Systems Handbook,” Academic Press Inc., 1994.

All of the above are incorporated herein by reference.

New and improved systems and methods are needed to reduce or eliminatedangerous driving situations involving cognitive distractions includingdistracting conversations with vehicle passengers or the use oftelecommunication devices or cellular telephones for voicecommunications or texting and that also take into account dangerousdriving conditions including, for example, dangerous roads, trafficcongestion, pedestrian traffic, day/night driving, weather conditions aswell as motor vehicle condition including, for example, possible brakeproblems, tire problems, automotive engine problems or other motorvehicle problems that may contribute to increased accident probability.More particularly, none of the above prior art systems or methods makeuse of artificial intelligence with expert system analysis involvingcombinations of motor vehicle motion sensor technology, directional RFantenna technology, near field communications technology, microphonearray technology with acoustic beamforming and noise cancellation,speech-to-text technology, image analysis, driver medical emergencysituations and integration of vehicle telematics reporting vehiclesafety problems. What is needed is a totally integrated operationalelectronic system that takes advantage of these modern technologies toreduce motor vehicle accidents and injury to passengers and/orpedestrians resulting from driver cognitive distractions, dangerous roadand driving conditions, and motor vehicle problems while still ensuringa driver may use cellular telephone or other wireless communicationdevices to report medical conditions or critical driving situationsrequiring immediate assistance.

SUMMARY OF THE INVENTION

Various embodiments of systems and methods for improved warning ofdrivers of dangerous driving situations and for control of the use ofcellular telephones and/or other wireless telecommunication devices bydrivers of a moving vehicle are disclosed. In one aspect of thisinvention, electronic specifically programmed motor vehicle devicecontrol system and methods are disclosed, with at least one specializedelectronic communication computer machine including electronicartificial intelligence expert system decision making capability usingone or more motor vehicle electronic sensors capable of monitoringactivities of the motor vehicle and the driver and/or passengersincluding activities related to the use of cellular telephones and/orother wireless communication devices and further comprising electroniccommunications transceiver assemblies for communications with externalsensor networks to obtain information on weather conditions, roadwayconditions, pedestrian traffic and/or traffic congestion conditions andother dangerous situations wherein the electronic specificallyprogrammed motor vehicle device control system and methods make use ofartificial intelligence expert system decision making capability basedon the electronic sensor inputs to derive warning and control signalsfor warning the driver of dangerous driving situations and/or forcontrolling the motor vehicle driver use of the cellular telephoneand/or other wireless communication devices.

In a further aspect of some embodiments of the invention, the electronicspecifically programmed motor vehicle device control systems and methodselectronic sensors comprise near field communication sensors.

In a further aspect of some embodiments of the invention, the electronicspecifically programmed motor vehicle device control systems and methodselectronic sensors comprise noise reduction beamforming microphonearrays.

In a further aspect of some embodiments of the invention, the electronicspecifically programmed motor vehicle device control systems and methodselectronic sensors comprise interference reduction directional RFantennas.

In a further aspect of some embodiments of the invention, the electronicspecifically programmed motor vehicle device control systems and methodselectronic sensors comprise image cameras for use with image analysissoftware.

In yet a further aspect of some embodiments of the invention, theelectronic specifically programmed motor vehicle device control systemsand methods are specifically programmed to derive artificialintelligence expert system measures of road/weather indices basedexternal sensor network signals reporting road-way and/or weatherconditions.

In a further aspect of some embodiments of the invention, the electronicspecifically programmed motor vehicle device control systems and methodsare specifically programmed to derive artificial intelligence expertsystems driving warning indices based on the above derived road/weatherindices and external sensor network inputs reporting pedestrian and/ortraffic congestion conditions.

In still a further aspect of some embodiments of the invention, theelectronic specifically programmed motor vehicle device control systemsand methods are specifically programmed to derive artificialintelligence expert system driver warning indices combining results ofthe artificial intelligence expert system above derivation of thedriving warning indices with motor vehicle sensor inputs reportingdriver distractions from the use of wireless communication devices orcellular telephones or other driver or passenger distracting activitiesgiving rise to dangerous driver situations.

In yet another aspect of some embodiments of the invention, theelectronic specifically programmed motor vehicle device control systemsand methods derive driver warning indices based on fuzzy logiccalculations.

In still a further aspect of some embodiments of the invention, theelectronic specifically programmed motor vehicle device control systemsand methods use the results of the artificial intelligence expert systemand/or fuzzy logic calculations to lock out or inhibit the use of acellular telephone or other wireless communication device by the driverof the motor vehicle.

In a further aspect of some embodiments of the invention, the electronicspecifically programmed motor vehicle device control systems and methodsmay ignore derived cellular telephone or other wireless communicationdevice lock-out or inhibiting control to permit emergency communicationsby the driver of the motor vehicle such as the placing of “911” calls orother emergency calls requiring immediate assistance.

These and other aspects of the invention herein disclosed are disclosedbelow.

BRIEF DESCRIPTION OF THE DRAWINGS

While the present invention is amenable to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and will herein be described in detail. Theinventions of this disclosure are better understood in conjunction withthese drawings and detailed description of the preferred embodiments.The various hardware and software elements used to carry out theinventions are illustrated in these drawings in the form of figures,block diagrams, flowcharts and descriptive tables setting forth aspectsof the operations of the invention.

It should be understood, however, that the drawings and detaileddescriptions are not intended to limit the invention to the particularform disclosed, but on the contrary, the intention is to cover allmodifications, equivalents and alternatives falling within the spiritand scope of the present invention as defined by the appended claims.

FIG. 1 illustrates, without limitation, a moving vehicle interiortypical of the moving vehicle interior environment of this invention.

FIG. 2 illustrates, without limitation, a sensor and controlconfiguration of this invention.

FIG. 3 depicts, without limitation, in block diagram form a devicecontrol unit with capabilities useful for this invention.

FIG. 4 illustrates, without limitation, example placement of near fieldcommunication sensors in a vehicle interior.

FIG. 5 illustrates, without limitation, integration of near fieldcommunication sensors in an embodiment of this invention.

FIG. 6 depicts, without limitation, acoustic beamforming arraymicrophones useful to isolate and detect audio speech from the driver ofa moving vehicle.

FIG. 7 depicts, without limitation, a representative frequency responseof an acoustic beamforming array microphone as a function of the angleof an impinging acoustic waveform.

FIG. 8 depicts, without limitation, a directional RF (radio frequency)antenna useful in monitoring the use of a telecommunication device bythe driver of a motor vehicle.

FIG. 9 depicts, without limitation, typical directional RF antenna areacoverage patterns as a function of angular displacement of the antenna.

FIG. 10 depicts, without limitation, a camera to capture an image of amoving vehicle driver useful in determining the use of atelecommunication device by the driver.

FIG. 11A depicts, without limitation, a partial flowchart for theoperations of this invention.

FIG. 11B depicts, without limitation, a partial flowchart for additionaloperations of this invention continuing from FIG. 11A of this invention.

FIG. 12 depicts, without limitation, exemplary artificial intelligenceexpert systems sensor decision matrix for motor vehicle driver activitysensors of FIG. 11B.

FIG. 13 depicts, without limitation, exemplary use of a generalartificial intelligence fuzzy logic controller with roadway, traffic,weather and driver distraction input variables for analysis to produceoutput driver warning and control signals.

FIG. 14 depicts, without limitation, a flow chart for artificialintelligence analysis of the input variables of FIG. 13 to produce theoutput driver warning and control signals.

FIG. 15A depicts, without limitation, an exemplary artificialintelligence expert system decision matrix for a road-weather warningindex based on dangerous roadway and weather conditions on roadways orhighways being traveled by the motor vehicle.

FIG. 15B depicts, without limitation, an exemplary fuzzy logic analysisof the expert systems artificial intelligence road and weatherconditions matrix of FIG. 15A to produce a crisp single road/weatherwarning index.

FIG. 15C depicts, without limitation, an exemplary artificialintelligence expert system decision matrix for a vehicle driving warningindex based on traffic and the derived road-weather warning index ofFIG. 15B.

FIG. 15D depicts, without limitation, an exemplary fuzzy logic analysisof the expert systems artificial intelligence road/weather and trafficconditions matrix of FIG. 15C to produce a crisp singleroad/weather/traffic vehicle driving warning index.

FIG. 15E depicts, without limitation, an exemplary artificialintelligence expert systems decision matrix for a driver warning indexbased on driver distraction and the vehicle driving warning index ofFIG. 15D.

FIG. 15F depicts, without limitation, an exemplary fuzzy logic analysisof the expert systems artificial intelligence of the vehicle driving anddriver distraction expert systems matrix of FIG. 15E to produce a crispsingle driver warning index based on the combined analysis of roadway,weather, traffic and driver distraction input variables.

FIG. 15G depicts, without limitation, an alternative exemplary fuzzylogic analysis of the expert systems artificial intelligence of thevehicle driving and driver distraction expert systems matrix of FIG. 15Ewith a lower driver distraction index than that of FIG. 15F to produce acrisp single driver warning index based on the combined analysis ofroadway, weather, traffic and driver distraction input variables.

FIG. 15H depicts, without limitation, the partial flow of information inthe expert artificial intelligence decision-making set forth in FIGS.15A-15G.

FIG. 16 depicts, without limitation, an exemplary fuzzy logic controlleruseful in the present invention.

FIG. 17 depicts, without limitation, an exemplary flowchart for lock-outof wireless device control logic considering derived and artificialintelligence expert system driver danger warning index together withemergency driver medical conditions and/or critical driving situations.

FIG. 18 depicts, without limitation, exemplary warning graphics forpossible display to the driver.

DETAILED DESCRIPTION

The above figures are better understood in connection with the followingdetailed description of the preferred embodiments.

System and Methods Description

FIG. 1 depicts, without limitation, a typical interior environment (101)of a moving vehicle (100) of the type addressed by this invention. Inthe depiction of FIG. 1, two passengers (103) and (105) are illustratedoccupying the front seats of the vehicle. The passengers are wearingseatbelts (107) indicating they are in a moving vehicle or intending tobe in the vehicle while it is moving. Other interior objects of themoving vehicle (100) useful in characterizing activities in the motorvehicle include, for example, the steering wheel (106).

This invention discloses apparatus and methods for determining when thedriver (103) of the moving vehicle (100) is using a telecommunicationdevice such as a cellular telephone for voice or text communicationwhile the vehicle is moving. The inventive systems and methods of thisinvention further include the capability of combining detected dangerousdriver activities with dangerous roadway, traffic and/or weatherconditions and/or other driver distractions to provide a comprehensivewarning of danger situations presented to the driver and occupants ofthe motor vehicle. At the same time, the apparatus and methods of thisinvention are conceived to permit the usage of a telecommunicationsdevice or cellular telephone by other passengers, such as passenger(105), of the moving motor vehicle (100) who are not driving thevehicle.

As depicted in FIG. 1, a device control unit (102) monitors the interiorenvironment (104) of the moving vehicle (100) to detect dangerousactivities and situations including determination of when the driver(103) is using a telecommunications device such as a cellular telephoneor other wireless device while driving and further for issuing drivingwarnings and/or transmitting control signals to inhibit such use whilethe vehicle is moving. The same device control unit (102) is configuredto isolate such control to the driver (103) while permitting use oftelecommunication devices or cellular telephones by passengers (104).While the environment of FIG. 1 is shown with only two passengers, thesame device control unit (102) may be configured to provide the samewarnings and/or inhibiting control of the driver use of suchtelecommunication devices in vehicles with more than two passengers.

FIG. 2 depicts, without limitation, a motor vehicle sensor and controlconfiguration (200) useful in the present invention. The configuration(200) includes the device control unit (102) of FIG. 1. The devicecontrol unit (102) is connected to motor vehicle occupant activitysensors (206) for the purpose of monitoring activities of the driver andpassengers within the motor vehicle. The purpose of the sensors (206) isto determine activities within the motor vehicle that might give rise toa distraction to the driver that results in a dangerous drivingsituation. Those activities may include the use of a cellular telephoneor other wireless device for talking or texting and/or activities ofother passengers in the motor vehicle that may be distracting to thedriver. The occupant sensors (206) may include Near Field Communication(NFC) sensors, microphone sensors, cameras sensors, RF signal sensors orother sensors useful to monitor passenger activities.

As shown in FIG. 2, the device control unit (102) also communicates withthe motor vehicle telematics database and programs (201). The telematicsunit (201) is also interfaced to motor vehicle displays (203) includingdashboard displays, gauges and screen displays used to inform the driverof automobile of driving conditions as well as to provide interface andcontrol to other vehicle systems such as entertainment, climate control,navigation, communication and other motor vehicle systems. The motorvehicle telematics unit (201) also interfaces to multiple motor vehicleoperational sensors (202) including for example, tire pressure sensors,braking system sensors, fuel sensors, oil sensors, temperature sensors,vehicle maintenance warning sensors, lighting system sensors, headlighttaillight and braking system sensors, turn signal sensors, exteriorcamera and proximity sensors, road tracking sensors, erratic drivingbehavior sensors, location sensors, weather sensors and any othersensors used to monitor motor vehicle status and driving conditions.

As further illustrated in FIG. 2, the motor vehicle telematics unit(201) connects to the motor vehicle communications system (204) forcommunications with remote information and control systems. The motorvehicle communication system (204) communicates via radio frequencysignals with antenna (205) with these remote systems. Such systems mayinclude traffic control systems, weather reporting systems, roadconditions systems, police or law enforcement systems, emergency alertsystems and the like.

An additional roadway traffic safety concern involves pedestrians beingstruck by motor vehicles. A report issued by the Governors HighwaySafety Association (GHSA) estimates the number of pedestrian fatalitiesjumped 10% in 2015 after a 19% increase from 2009 to 2014. SeePedestrian Traffic Fatalities by States—Preliminary Data, 2015, GHSA,June 2015. That report estimates that for the first time in 25 years,pedestrian deaths in 2015 are projected to account for 15% of trafficfatalities. It has been reported that a 2010 study indicated thatpedestrian fatalities caused by distracted motor vehicle driversincrease by 50% over five years. Factors of concern include pedestriantraffic density with an increased number of pedestrians being present atspecial events such as concerts and sporting events, increase pedestriantraffic in specific locations such as school zones, shopping districts,parks, business districts or other areas with increased number of peoplewalking around. Another factor is the rise in “walkable communities”with more people walking or bicycling to work. See Dallas Morning News,Mar. 9, 2016, page 9A. Warnings of dangerous pedestrian trafficsituations may be reported to the motor vehicle telematics unit (201) bythe motor vehicle communications Center (204) of FIG. 2. In this waysuch pedestrian traffic information is made available for use inevaluating roadway/traffic dangerous factors in the overall driverwarning calculations of this invention.

Roadway/traffic monitor unit (207) interfaces with multipleroadway/traffic sensors (209) as also shown in FIG. 2. Theroadway/traffic monitor unit (207) may be consolidated as part of acomprehensive roadway and traffic monitoring system or may representmultiple such monitoring systems including law enforcement monitoringsystems, roadway monitoring systems, traffic monitoring systems,accident reporting systems, roadway construction systems and other suchsystems that may be employed to monitor roadway and traffic conditions.The results of such monitoring may be transferred by one or more antennasystems (208) to motor vehicle communication units (204) by antenna(205). The results of such exterior roadway and traffic monitoring maybe used in the artificial intelligence expert system embodiments of thisinvention as described further below.

FIG. 3 depicts, without limitation, a block diagram of possible elementsof an exemplary device control unit (300) corresponding to devicecontrol unit (102) of FIG. 1. The device control unit (300) of FIG. 3depicts a comprehensive collection of possible capabilities of thedevice control unit (102) of FIG. 1. It is to be understood that thedevice control unit (102) of FIG. 1 and as described elsewhere in thisspecification or in different embodiments of this invention may includeall or a selected subset of the total capability of the device controlunit (300) of FIG. 3.

The processor (301) may be of any suitable configuration known to thoseof skill in the art. For example, the processor (301) may be a computer,microprocessor, a DSP (digital signal processor), or other controlcircuitry suitable for this application. In addition, the processor(301) may be configured using a combination of these technologies.

As shown in FIG. 3, the device control unit may include multipleinterconnected capabilities that may be attached to or designed as anintegral part of the hardware or software of the processor (301). Thesevarious capabilities useful in the operation of the device control unitof this invention are characterized below and discussed more completelybelow in association with the additional FIGS. 4-10.

As indicated in FIG. 3, the device control unit (300) may includehands-free unit (315) permitting operation of a telecommunicationsdevice or cellular telephone in a hands-free mode. Such units mayconnect to a telecommunications device or cellular telephone using, forexample conventional Bluetooth or Bluetooth Low Energy (BLE) (310),Wi-Fi (311) or other radio frequency data transceiver (309)communication links. The hands-free unit (315) permits answering,placing and carrying on voice or text communications via an externalcellular telephone network using voice commands only without requiringthe driver to hand-manipulate or operate telecommunications or cellulartelephone equipment while driving.

As indicated in FIG. 3, the device control unit may also includeconnections to near field communication (NFC) sensors (302) to assist indetermination that the driver of the motor vehicle is using a wirelesscommunication device such as a cellular telephone. As explained furtherbelow NFC sensors may be used to detect the use of a cellular telephonein close proximity to that sensor, typically within 10 cm to 20 cmdistance. Such sensors operate through magnetic induction and are beingimplemented today in a variety of devices to detect the presence ofitems of interest and to exchange information with such items for theparticular purposes being served.

As also indicated in FIG. 3, the device control unit (300) may includeone or more directional, beamforming microphone arrays (303). Suchdirectional, beamforming microphone arrays are useful in isolating andcapturing audio voice signals from individual speakers in the presenceof interfering signals from other speakers and other environmental noisesignals. For example, in the environment depicted in FIG. 1,environmental noise signals may include audio signals generated fromother sources including other passengers, radio, automotive engine andvehicle operation and external noises such as generated by traffic orwind outside of the vehicle or other road noises. Directionalbeamforming microphone arrays are particularly useful in isolatingspeech signals of a desired speaker to the exclusion of other noisesignals in the environment of the speaker.

As also shown in FIG. 3, the device control unit (300) may include aspeech-to-text conversion capability (304). In some embodiments of thisinvention the speech-to-text conversion capability (304) may be used toconvert speech signals received from the directional microphone arrays(303) to text form, as well as for conversion of speech signals receivedby the device control unit (300) from the telecommunication device orcellular telephone being used by the vehicle driver to for texting. Thedevice control unit (300) may compare the converted text form of speechsignals received from the directional microphone arrays with thosereceived from the telecommunication device cellular telephone as part ofthe verification that the driver is indeed using those communicationdevices while driving as explained further below. Also, in someembodiments, the speech/text conversion capability (304) may be used toconvert text information or messages to speech enabling communicatingwith the driver of the motor vehicle or others in the motor vehicle inan audible, recognizable speech format. This capability may be importantin some embodiments for system control and providing audibleinstructions or warnings to the driver and/or other occupants of themotor vehicle, informing them of dangerous situations or activities.

As also shown in FIG. 3, the device control unit (300) may include anoptical camera (305) for capturing images of the driver of a movingvehicle to assist in the determination of situations where the driver isusing or attempting to use a telecommunications device such as acellular telephone while the vehicle is moving. In some embodiments ofthis invention, the optical camera (305) may be capable of takingmultiple individual photos or videos encompassing the driver of thevehicle in isolation or other areas of the vehicle. For example,photographing or taking videos of other areas of the vehicle may be ofassistance in determining the presence or absence of other passengers inthe vehicle and ascertaining their conversational activities and/ortheir use of separate telecommunications devices or cellular telephones.

The optical cameras (305) of FIG. 3 may also be used with image analysissoftware to perform such tasks as facial recognition to identifyparticular drivers. The device control unit (102/300) may be used tomaintain history files of motor vehicle drivers with histories of theirdriving habits including dangerous driving tendencies. Particulardrivers present increase risks for automobile accidents including forexample teenagers and especially teenage boys. Facial recognition may beused to assist in identifying drivers.

The optical cameras (305) of FIG. 3 may also be used to analyze specificmotions of the driver of a motor vehicle including, for example, raisingof a cellular telephone or other wireless device to the driver's ear foruse. Such optical cameras may make use of motion sensitive detectioncapability such as used today in video game systems.

As also shown in FIG. 3, the device control unit (300) may include an RF(radio frequency) directional antenna and receiver (306) to receive anddetect radio signals emanating from the area of the driver of a movingvehicle. For example, such signals may originate as cellular telephonesignals, Bluetooth signals, Wi-Fi signals, or other communication orcontrol signals. This same RF (radio frequency) directional antenna andreceiver (306) may also be used to selectively transmit such signalsinto the selected area occupied by the driver of a moving vehicle.

As further indicated in FIG. 3, the device control unit (300) mayinclude a GPS (Global Positioning System) receiver (307) useful fortracking the location and movements of the motor vehicle. The GlobalPosition System (GPS) makes use of triangulation calculations ofpositions based on signals received, for example, from multiplegeostationary satellites. Such systems provide location informationaccurate within approximately one meter. Massive databases existproviding GPS coordinates for virtually every addressable location inthe United States and elsewhere. Mobile communication networks implementHome Location Registries (HLRs) and Visitor Location Registries (VLRs)providing instant location information for mobile wireless devicesthrough the country. Such databases also provide detailed maps ofhighways and roadways used by motor vehicles. Such route maps andlocation information may be used in the present invention to verify thatthe motor vehicle is indeed traveling on established highways orroadways and further to provide markings of the location of such avehicle as a function of time along those known routes. This informationcan be used in combination with accurate time/clock informationavailable to the device control unit (300) of this invention using, forexample, time/clock distribution unit (316) shown in FIG. 3. Knowing theelapsed time interval between successive points with known distancebetween those points permits computation of the speed of the movingmotor vehicle. This information may, in-turn, be used for comparisonwith pre-established motor vehicle speed thresholds and predeterminedlocations. The results of such comparisons can be used by the devicecontrol unit (300) of this invention to determine when to initiatecomputations and processing prior to issuing a warning to the driver ofdanger or a command that inhibits operation of a telecommunicationsdevice or cell phone by the driver of a moving vehicle.

Motor vehicle location information may also be derived based on thevehicle distance from cellular telephone towers or other known fixedlocations transmitting signals that may be received by one of thereceivers of the device control unit (300) of FIG. 3. Here again,triangulation calculations may be made using three or more such locationtransmission signals.

As also shown in FIG. 3, the device control unit (300) may include acellular transceiver (308) used to receive and transmit cellularcommunication information between the device control unit (300) andexternal sources accessible to the cellular telephone network or atelecommunication device or cellular phone located in the motor vehiclebeing used by a passenger or the driver of that vehicle.

In addition, as shown in FIG. 3, the device control unit (300) mayfurther include a data transceiver (309) useful for communications withother devices in the motor vehicle including vehicle informationsystems, control and display systems, as well as telecommunicationdevices or cellular telephones used by passengers or the driver of themotor vehicle.

Similarly, as shown in FIG. 3, the device control unit (300) may alsoinclude a Bluetooth transceiver (310) and/or a Wi-Fi transceiver (311).Both Bluetooth and Wi-Fi transceivers are used for short range voice anddata communications. In the present invention such transceivers may beused to communicate between device control unit (300) and thetelecommunication device or cellular telephone used by the driver orother occupants of the motor vehicle. Such communications may be used tomonitor transmit and receive signals and to provide full-duplex controlcommunication between the device control unit (300) and thetelecommunication device of interest. Transceivers (309), (310) and(311) may also be used to communicate with the motor vehicle operationalsensors (202) of FIG. 2. Those operational sensors include, withoutlimitation, near-field communication devices, directional microphonearrays, RF antennas, and cameras all used to monitor activities withinthe motor vehicle.

In some embodiments, the device control unit (300) of FIG. 3 may includeartificial intelligence expert system technology (312) with the goal ofimproving decisions made by the device control unit (300). Suchartificial intelligence expert system technology may prove especiallybeneficial in assessing the degree of danger and various situations. Forexample, danger may be higher when the motor vehicle is operated athigher speeds or with dangerous vehicle maneuvers. Other considerationsin assessing such degrees of danger may include driving conditions suchas whether conditions involving for example, rain, sleet, snow, ice orwind. In addition, artificial intelligence expert system technology maybe used to incorporate traffic considerations such as trafficcongestion, dangerous roadways such as narrow roads or switchbackmountainous roads. Other considerations may include danger arising fromroad or highway construction activities. In addition, known road orhighway problems such as the presence of potholes, dangerous curves,animal crossing areas or the like may be taken into consideration byappropriately designed artificial intelligence expert systems. Otherexamples of situations that may give rise to heightened dangers includeemergency problems such as those arising from accidents on the roadwayor highway being traveled, fires, criminal activities, or emergencyvehicle traffic including fire trucks, ambulances or police or othergovernment vehicles dispatched on an emergency basis to deal with crisissituations or pedestrian congestion resulting from large numbers ofpeople or crowds in the areas being traveled by the motor vehicle. Suchroad way or highway conditions, weather alerts or other dangeroussituations are frequently transmitted by appropriate government areroadway condition agencies for the purpose of alerting drivers ofpotentially dangerous situations. Such broadcast may be received forexample by cellular transceiver (308) or data transceiver (309) or otherappropriate receivers designed to receive such signals.

The artificial intelligence expert system capability (312) may alsoinclude “learning” capability, including the development of databasesrecording driving habits of particular drivers, such as driving acumenand the ability to react to particular potentially dangerous situations.

As also indicated in FIG. 3, the device control unit (300) may includefuzzy logic capability (313). Fuzzy logic is a method of representinganalog processes on a digital computer. With fuzzy logic control,statements are written in the form of the propositional calculus logicstatements. These statements represent somewhat imprecise ideasreflecting the states of the variables. Fuzzy logic is particularlyappropriate when an expert is available to specify these propositionalstatements characterizing the relationships between system variables. Inthe present invention such propositional statements and fuzzy logic maybe beneficial in analyzing the relationships between various parameterscharacterizing dangerous situations and responses to those situations asdescribed more completely below.

Telecommunication device or cell phone “pairing” (314) may also beincluded in the device control unit (300) of the present invention. Such“pairing” permits a telecommunication device or cell phone to beconnected to device control unit (300) by telecommunication links suchas Bluetooth, Wi-Fi or the like. With these connections, voice or datacommunication signals transmitted to and from the telecommunicationsdevice or cellular telephone may be relayed through the device controlunit (300) via the interconnecting telecommunication links. In addition,such “pairing” permits commands and responses to be communicated betweena telecommunications device or cellular telephone and the device controlunit (300). One intended use of such commands would be to inhibitoperation of the telecommunications device or cellular telephone indangerous situations.

In addition, as shown in FIG. 3, the device control unit (300) mayfurther include a data-base access capability (325) connected toprocessor (301) for accessing and updating data-base information usefulin the operation of the present invention. The data-base information maybe stored locally as part of the device control unit (300), or maybelocated remotely and accessible, for example, from the cloud through theInternet or cellular telephone communication networks. In someinstances, database information may also be accessed from informationstored and in other control and information data files implemented inthe motor vehicle such as information stored for use by vehicleinformation display systems. Such vehicle information display systemsmay include information necessary for dashboard displays concerningvehicle operational status, speed, odometer readings, engineperformance, fuel levels and warning signals. In addition, other controland information data files implemented in the motor vehicle may includefiles used to drive other on-board displays including, for example,touch screen displays or displays manipulated using point-and-click orother operator controls for navigating and selecting information to bedisplayed including, for example, navigation information and maps,vehicle status, weather, entertainment system control, telecommunicationdevice control and the like. In some embodiments of this invention,information from device control unit (300) may in fact be displayed onsuch other on-board displays or may be made available for access by themotor vehicle driver passengers using such displays. In some embodimentsof this invention the device control unit (300) may be integrated intoand made an operational part of other vehicle control and/or displaysystems including, for example, the motor vehicle telematics unit 201 ofFIG. 2.

In addition, as shown in FIG. 3, the device control unit (300) mayfurther include a time/clock distribution capability (316) operating tomake accurate date and time information available to the device controlunit (300). Such information may be used, for example, in thecalculation of vehicle speed by providing elapsed time betweenparticular vehicle location points along a route of travel. Such timeand date information may also be used to create history files recordingoccurrences of driver use of telecommunication devices or cellulartelephones while driving the motor vehicle. In some embodiments suchinformation may be shared with police or other government agencies toreport dangerous driving activities of motor vehicle drivers. In someembodiments such information may be reported to vehicle owners oroperators such as taxicab companies, trucking companies, rental caragencies or other equipment leasing or renting organizations employingor otherwise using drivers to operate their vehicles or equipment. Suchreported information may be used, for example, to work with drivers toimprove their driving habits, admonish drivers for bad behavior or totake other corrective action deemed necessary for safety or liabilityconcerns.

In addition, as shown in FIG. 3, the device control unit (300) mayfurther include accelerometer (317) capabilities. An accelerometer is adevice that can measure the force of acceleration, whether caused bygravity or by movement. An accelerometer can therefore be used tomeasure or assist in the measurement of the speed of movement of anobject to which it is attached. Because an accelerometer senses movementand gravity, it can also sense the angle at which it is being held. Forexample, solid-state accelerometers can sense the tilt, movement andspeed being applied to them, and are commonly used for video gamesystems. As explained further below, such tilt sensitive accelerometersmay be used to sense the orientation of driver telecommunication devicesor cellular telephones to further assist in detecting dangerous use ofsuch devices by the driver of a moving vehicle. Useful accelerometertechnology includes piezoelectric, piezoresistive, resonant,strain-gauge, capacitance, tunneling, and heated liquid and gasaccelerometers. Silicon MEMS accelerometers that work on the capacitiveapproach or ones that that are based on temperature differentials inheated-gas are useful in some embodiments of this invention. Suchthermal accelerometers may be fabricated in monolithic structures withintegration with all the necessary signal conditioning, interface andembedded circuitry on a single integrated circuit. Accelerometers areused today in automobiles for crash detection and airbag deployment anddetection of automobile rollover accidents.

A speaker unit (318) may also be included as part of the device controlunit (300). The speaker may be used to announce warnings or otherwiseinform the driver or passengers in the vehicle of dangerous situationsdetected by device control unit (300). The speaker may also be used toinstruct the driver or passengers using particular telecommunicationdevices or cellular telephones in the motor vehicle to discontinue useof those devices. The speaker may also announce that, for safetyreasons, the device control unit (300) has taken automatic actions todisable particular telecommunication devices or cellular telephonesbeing used in the motor vehicle.

In addition, as shown in FIG. 3, the device control unit (300) mayfurther include associated memory (319) for storing software programs,vehicle information, measurement history information and other datauseful or collected by the device control unit (300) in the operationsof this invention. The associated memory (319) may comprise randomaccess memory (RAM), read only memory (ROM), solid-state memory, diskmemory, optical memories or any other appropriate memory technologyknown to those of skill in the art. While memory unit (319) is shown inFIG. 3 as a separate assembly, it is to be understood that some or allof such memory may be distributed among the various operational, controland communication capabilities illustrated in FIG. 3.

In addition, as shown in FIG. 3, the device control unit (300) mayfurther include display capability (320) for displaying operationalstatus and information concerning the operation and calculated resultsderived by the device control unit (300). Such information may includeinformation reflecting potentially dangerous usage by the driver of themotor vehicle of telecommunication devices or cellular telephones whilethe vehicle is moving. The display (320) may be a separate displayassociated with device control unit (300), or, alternatively, thedisplay (320) may be integrated with an operational part of otherdisplays present in the motor vehicle including those discussed abovesuch as the motor vehicle telematics unit 201 of FIG. 2. Useful displaytechnologies include liquid crystal displays (LCD), light emitting diodedisplays (LED), plasma displays, smart glass, touch screen displays,menu-driven displays, and displays operated using speech commands orother suitable display technology.

In addition, as shown in FIG. 3, the device control unit (300) mayfurther include additional input-output-device (321) capabilities. Forexample, standard USB ports may be used for such access. Otherpossibilities include the Common Flash memory Interface (CFI) approvedby JEDEC and other standard memory unit interfaces. Other possibilitiesinclude audio input/output ports, video ports such as HDMI ports andother input/output capabilities.

As also shown in FIG. 3, the device control unit (300) may furtherinclude an RFID (radio frequency identification) tag device (322) usedto identify the motor vehicle and communicate information or resultsfrom device control unit (300) to RFID tag readers located alonghighways tollways or roadways along which the motor vehicle istraveling. The RFID tag device (322) operating with device control unit(300) may be powered by power supply (323) of the device control unit(300) shown in FIG. 3 and discussed above. Alternatively, the RFID tagdevice (322) may be powered from externally generated electromagneticenergy waves emitted by an RFID tag reader. Information transmitted fromthe RFID tag device (322) may include information indicating dangeroususage of telecommunication devices or cellular telephones by the driverof the motor vehicle.

In addition, as shown in FIG. 3, the device control unit (300) mayfurther include a power supply (323) necessary for operation of thedevice control unit (300) including the various capabilities depicted inFIG. 3. The power supply (323) may derive energy from the vehicleelectrical power supply source or may be implemented as a separatebattery or energy supply including, without limitation, solar energy,energy derived from external impinging electromagnetic waves, or energyderived from motor vehicle mechanical operations such as breaking orcoasting.

As also shown in FIG. 3 device control unit (300) may include anomnidirectional antenna (324) useful for receiving RF signals from alldirections as opposed to being limited to particular directions such asthe case with the RF directional antenna and receiver (306) discussedabove. For example, the omnidirectional antenna (324) may be useful inmonitoring the use of telecommunication devices or cellular telephonesby passengers of the motor vehicle other than the driver of thatvehicle. As explained further below, in dangerous driving situations itmay be appropriate to limit such use by other passengers because ofdistractions that may be caused to the driver of the vehicle. In othercircumstances it may be appropriate to permit such usage by otherpassengers of the vehicle.

FIG. 3 also depicts the connection of driver medical sensors (326) withthe device control unit (300). Such sensors may be wearable and used tomonitor driver vital signs and general health conditions when drivingthe motor vehicle. Modern medical sensor technology includes short rangetelecommunications capability that enables transmission, detection andanalysis of medical sensor signals derived from such wearable devices.Such sensors may be implemented in accordance with Internet of Things(IoT) technology and implementations being employed today in severaldistributed control and monitoring applications. IoT devices generallymake use of short-range radio frequency communication capability such asBluetooth, Low Energy Bluetooth (BLE), Wi-Fi, ISM or other suchcommunication technologies. Conditions such as heart rate, bloodpressure, breathing parameters, asthma conditions, incapacitation and/orother critical driver medical condition parameters may be monitored ofthe medical sensors (326). Such sensors may be configured for example,without limitation, as medical sensor cuffs, patches, implants, wristbracelets, ankle bracelets, eyeball activity and/or condition sensors orother medical sensor technology implementations. Sensors used to monitordriver sobriety including alcohol intoxication or indication of the useof drugs such as marijuana, heroin and the like may also be includedwith the medical sensors (326) of FIG. 3. Driver intoxication fromalcohol or drug use gives rise to impairment of judgment, reaction timeor ability to deal with dangerous situations and is a major cause ofautomobile accidents.

As explained further below, in certain medical emergency situations itmay be unacceptable or inappropriate to block the driver usage ofcellular telephone or wireless device technologies for the purpose ofsummoning help or assistance during such medical emergency. Also,detection of a medical emergency situation may be the basis for issuinga warning to the driver of a dangerous medical condition and advisingthe driver to safely bring the motor vehicle to a halt and to seekmedical assistance. For these are reasons, integration of the outputs ofmedical sensors (326) into the overall artificial intelligence expertsystem warning and control system herein described may be an importantconsideration in the implementation of such a system.

It is to be understood that while the device control unit (300) of FIG.3 is depicted and described above as a unitary assembly, it is alsopossible, and in some cases desirable, that perhaps some of theoperational features shown in FIG. 3 are shared and possibly implementedas part of other automobile control, communications, processing and/ordisplay capabilities such as the motor vehicle telematics unit (201) ofFIG. 2. In addition, it should be clear that several of the operationalcapabilities of the device control unit (300) of FIG. 3 may beimplemented with distributed devices and/or capabilities locatedthroughout the motor vehicle and communicating with the processor (301)as indicated in FIG. 3.

It should further be understood that other embodiments of the systemsand methods of this invention may use a subset of the capabilitiesdepicted in FIG. 3 without departing from the fundamental integratedsystem and method teachings of this invention.

Sensor Systems and Methods Description

FIG. 4 depicts, without limitation, the possible location of proximitysensors such as Near Field Communication (NFC) proximity sensors todetect particular electronic devices such as cellular telephone, tabletcomputers or other electronic communication devices being used byparticular passengers in particular locations in the motor vehicle(400). For example, proximity sensor (401) may be located at the reartop of the driver's seat in close proximity to the driver's head todetect possible use of such electronic devices by the driver. Similarly,proximity sensor (402) may be placed at the top of the front passengerseat to monitor the use of such electrical devices by that passenger.Other possible locations of such sensors include the motor vehicleinterior roof or head-liner near the head of the driver and/or passengerillustrated by proximity sensors (403) and (404) to detect the use ofsuch electronic devices. Texting or other manual use of handheld devicessuch as cellular telephones and/or laptop computers are generallycarried out in or near the lap of the driver or passengers. The locationof proximity sensors, such as NFC sensors, may include the door sidepanel proximity sensor (405) or proximity sensor (406) located in thesteering wheel to detect operation of such electronic devices located inthe general lap area of the passenger or driver. Similar sensorlocations, not shown, may be used to monitor activities of passengers inother seats of motor vehicles.

FIG. 5 depicts, without limitation, an embodiment (500) involving NFCwith a mobile device (501) coupled by magnetic induction (504) to aproximity sensor (503) of the type illustrated in FIG. 4. NFC is a setof short-range wireless technologies, typically operating at aseparation of 10 to 20 cm or less. NFC standards specify operation at3.56 MHz with data rates from 106 Kbits/sec to 424 Kbits/sec. The NFCtag may be a read-only device or may also include writable memory. NFCuses magnetic induction (504) between two loop antennas forming an aircore transformer as illustrated in FIG. 5. NFC technology is included inmany smart phones available today. NFC is used, for example, forpersonal data storage such as debit or credit card information, loyaltyprogram data, Personal Identification Numbers (PINs), etc.

In the embodiment of FIG. 5, the NFC reader is incorporated in themobile wireless device (501). The NFC reader may be activated when theuser of the mobile device is attempting to make a cellular telephonecall or otherwise use that device. When activated, the NFC reader willdetect the proximity sensor mounted in the motor vehicle as describedabove. The mobile device may report the detection to the device controlunit (102/300) via RF signals (506) transmitted and received viaantennas (502) and 505) as illustrated in FIG. 5. The device controlunit (102) may then determine which of the multiple NFC proximitysensors located in the motor vehicle has been detected. If that sensoris associated with the driver of the motor vehicle, a danger warning maybe issued or the mobile device may be disabled and not allowed tooperate. If the sensor detected is located elsewhere in the motorvehicle indicating that it is a passenger other than the driver usingthe device, the device may be allowed to operate.

As also shown in FIG. 5, the proximity sensor may include, for example,a communication interface between the sensor and the device control unit(102/300). Without limitation, that communication interface may beimplemented, for example, as a Bluetooth Low Energy (BLE) communicationlink between the proximity sensor and the device control unit (102). BLEis a low-power version of Bluetooth that was designed for the Internetof Things (IoT). BLE may have a range up to about 200 feet dependentupon the capabilities of a particular device. BLE operates with about afour-year battery life of a coin size battery cell. Classic Bluetooth,Wi-Fi or other suitable radio technology may also be used for thecommunication links (506) of FIG. 5. As described above in FIG. 2, thedevice control unit (102) may also be connected to the motor vehicletelematics unit as indicated at (507) of FIG. 5.

FIG. 6 illustrates in area (601) without limitation, the use ofdirectional beamforming microphone array technology (605) and optionalarray (606) to isolate audio signals from the driver (602) to theexclusion of other sources of noise and interference present in themotor vehicle. A directional beamforming microphone array such as array(605) is designed and configured to be more sensitive to audio soundwaves arriving at the array from particular directions and lesssensitive to audio sound waves arriving from other directions. In otherwords, the directional beamforming microphone array (605) is mostsensitive to audio waveforms arriving in front of the array andcontained primarily within a defined conical area depending on thenumber of microphone sensors in the array, their geometricconfiguration, and array signal processing algorithms implemented in thedevice control unit (102/300) of FIG. 1 shown more completely in FIG. 3and discussed in the above paragraphs.

In addition to the microphone array and associated signal processingalgorithms using beamforming to eliminate, reduce or minimizeinterference and noise from other sources in the motor vehicle (600),such signal processing algorithms may also be designed to eliminate suchbackground noise and interference using additional time domain and/orfrequency domain audio signal analysis to further separate audiowaveforms from the driver (602) and other sources of background noiseand interference. Techniques for achieving such isolation of the desiredaudio signals from undesired background noise interference aredescribed, for example, in publications cited above and incorporatedherein by reference including: (1) M. Brandstein and D. Ward,“Microphone Arrays,” Springer, Berlin, Germany and New York, 2001; (2)J. Benesty, et. al., “Microphone Array Signal Processing,” Springer,Berlin, Germany and New York, 2008.

Multiple sources of potential background noise and interference areillustrated in FIG. 6 including, without limitation, wind noise from thewindows (611) and (612) of vehicle (600) indicated in the figure bysound waves (609) and (610). Additional sources of background noise andinterference (613) includes noise generated by the motor vehicle (600)itself including road noise, engine noise, entertainment system noiseand speaker noise (608) generated by other passengers such as passenger(603) talking while the driver (602) is using the telecommunicationdevice or cellular telephone (607).

In some embodiments of the present invention, a single directionalbeamforming antenna array may be used to sufficiently isolate the audiosound waves (604) from the driver (602) speech from other backgroundnoise and interference as described above. As described above thedirectional beamforming antenna array (605) of FIG. 6 will operate withmore gain in the direction of the speaker driver (602). However, it isalso true that sound ways from other sources including, for example,sound waves from a speaking passenger (603) will be received at thedirectional beamforming antenna array (605), albeit with reducedsensitivity. In one embodiment of this invention, apparatus and methodsare disclosed that permit comparison of speech signals from the driver(602) to those emanating from an exemplary passenger (603). Suchcomparisons may further enhance separation of these two or more signalsand further confirm that the speech signal being received at thedirectional beamforming antenna array (605) is indeed audio signals fromthe driver (602) of the motor vehicle (600). Such further isolation ofthe driver (602) speech signals from those of other passengers willensure that only the use of a telecommunications device or the cellulartelephone (607) by the driver is used for artificial intelligence expertsystems decision making as described further below while still allowingconversations and speech signals from other passengers. This approachmay be used to permit other passengers to use their telecommunicationdevices or cellular telephones.

In some embodiments of the present invention, the directionalbeamforming antenna array (605) may be capable of scanning the area ofinterest using this digital signal processing to focus the beam of thearray antenna in different directions. For example, in some embodimentsof the present invention, it may be desirable to obtain improved voicesignal quality from a passenger such as passenger (603) for the purposeof comparing those speech signals to those speech signals emanating fromthe driver (602) of the vehicle. Such beam scanning is known in the artand described in the above references.

As an alternative, and some in embodiments, additional directionalbeamforming antenna arrays such as array (606) may be used to moreaccurately capture signals emanating from sources other than the driver(602). These additional signals from the supplementary directionalbeamforming antenna array (606) may be used in the signal comparisonoperations described above.

FIG. 7 provides an exemplary illustration (700) of the performance ofone configuration of a directional beamforming antenna array. See J.Benesty, et. al., “Microphone Array Signal Processing, Springer, 2008,page 58. As can be seen from the figure, in this example, attenuation ofsignals arriving from outside the main lobe of the directional antennais considerably greater than that of signals arriving within that mainlobe or conical area covered by the directional beamforming antennaarray and its associated signal processing algorithms. Indeed, thedesign of this particular antenna array has uniform responsecharacteristics across the voice frequency band of interest from 0 to 4kHz. As explained in the above cited reference this exemplary antennaarray response is for a least-squares (LS) broadband beam former with 10equally linearly spaced sensors 4 cm apart. Other implementations ofdirectional beamforming antenna arrays may also be used in the presentinvention including implementations that attenuate selected frequencybands more than other frequency bands within the analyzed signalspectrum.

FIG. 8 illustrates an embodiment of this invention (800) that makes useof a directional RF (radiofrequency) antenna (808) mounted in motorvehicle (801). The directional RF antenna is designed with radiationbeam coverage (807) limited to the area around the driver (802). In thisway the directional RF antenna (808) will be primarily responsive tosignals within the radiation beam pattern coverage (807) and lesssensitivity to radio signals emanating from other telecommunicationdevices or cellular telephones such as (805). The detection of a radiosignal from a telecommunications device or cellular telephone (804) bythe directional RF antenna (808) will be a further indication verifyingthat the driver (802) of the vehicle is indeed using that communicationsdevice. In the depiction of FIG. 8, the directional RF antenna (808) ismounted above and behind the motor vehicle driver (802) to morespecifically isolate radiation pattern coverage to the area around themotor vehicle driver (802). This mounting of the directional RF antenna(808) advantageously minimizes reception from signals emanating frombehind the driver or to the side of the driver. Of course, othermounting configurations are possible.

FIG. 9 illustrates exemplary radiation beam coverage patterns (900)typical of such patterns for directional RF antennas. Pattern (901)corresponds to coverage in the plane of the antenna at 0°. Pattern (903)corresponds to beam coverage perpendicular to the plane of the antenna.As can be seen in pattern (903) considerable attenuation is achievedoutside of a 60° cone as indicated by the circular coverage scale (902).

FIG. 10 illustrates the use of a camera (1006) mounted in the motorvehicle (1000) useful in the present invention for capturing images ofthe vehicle driver (1001) to further verify the drivers use of atelecommunications device or cellular telephone (1003) or (1004) whilethe motor vehicle is in motion. Image analysis software implemented inthe device control unit (102/300) of FIGS. 1 and 3 may be used toanalyze images of the driver to verify that the driver is holding thetelecommunications device or cellular telephone (1003) to his ear asillustrated in FIG. 10. In addition to the appearance of atelecommunications device or cellular telephone (1003) in the imagecaptured by the camera, the image most probably will also include thedriver's (1001) arm and hand holding the device (1003). Image analysissoftware used to analyze human motions such as found in video games maybe used to detect movement of the driver's arm holding a cell phone andraising that cell phone to his ear for use. Image analysis software mayalso be used, for example, to detect driver drowsiness or sleeping bymonitoring the drivers opening or closing of eyes or position of thedriver in his or her vehicle seat.

In some embodiments of this invention, owners of a particular vehiclemay want to register electronic images of themselves as principaldrivers in the vehicle. Such images may be captured, for example, as“selfies” or as other digital images that are captured and stored in thedevice control unit memories. Comparison of these captured images withimages of the actual driver will enable a device control unit todetermine the presence of an unauthorized driver or at least a drivernot recorded in the device control unit image history files. Such afinding may be used to transmit an alert to the vehicle owner indicatingthat a driver has been detected that is unknown to the driver controlunit. Such transmission may occur, for example, via cellular telephoneor Internet communications. In some embodiments, responses from theowner may be required to verify that the person driving the vehicle hasbeen authorized by the owner in spite of the fact that the image of thedriver is not in the device control unit database. If no suchconfirmation is received particular alerts may be transmitted toauthorities to be aware of the unauthorized driver. Such information mayalso be used in the artificial intelligence expert system expert systemwarning and control system as described below.

In some cases, when the driver (1001) is using the telecommunicationdevice or cellular telephone for texting, the actual device may bepositioned at the location indicated for device (1004) shown in FIG. 10.In some cases, the combination of the audio signals received from thedirectional beamforming antenna array discussed above together with theimage captured by the camera may be conclusive evidence that the driver(1001) is indeed talking over these telecommunications device orcellular telephone (1003) in which case the device control unit 102 ofFIGS. 1, 2 and 3 may issue warnings or commands to inhibit such usedepending on other driving conditions. In some embodiments of thisinvention it may be useful to limit the field of view (1005) of thecamera (1006) as illustrated in FIG. 10 to exclude extraneous imageelements such as passenger (1002) that may complicate the image analysissoftware.

System and Methods Operational Description

FIGS. 11A and 11B depict, without limitation, an exemplary flowchart(1100) for operation of the system and methods of the present invention.As shown in FIG. 11A the flowchart begins at connector “A” (1101).Program initiation occurs at the start block (1102) and includesinitiation of the various capabilities of the device control unit(102/300) of FIGS. 1, 2 and 3 including the various capabilitiesillustrated in FIG. 3 and discussed above with proper initiation ofinternal software/hardware registers and other data necessary for systemoperation. As indicated in FIG. 11A processing operations are carriedout in communication with the motor vehicle telematics processor anddisplay and the device control unit (102/300) including access todatabase entries of the telematics processor and device control unit(1105). Initiation through connector “A” may occur, for example, whenthe motor vehicle engine is started. Initiation may also occur based onvehicle motion or other parameters accessed from telematics processorand/or device control unit (1105) indicating potentially dangeroussituations or based on other driver or externally generated controlsignals such as signals from a remote-control center as indicated, forexample, in FIG. 2.

Once initiated, control is passed to block (1103) for processing ofmotion detection signals. Readings of motor vehicle speed from thetelematics processor and/or device control unit (1105) may be used. Suchreadings may be based, for example, on vehicle velocity signals used todrive the vehicle speedometer. In addition, as discussed above,indications of vehicle motion including vehicle velocity andacceleration can be determined, for example, using GPS locationinformation derived from signals received by GPS receiver (307) of FIG.3. The GPS signals may be used in combination with time/clock signals(316) as also illustrated in FIG. 3. For example, knowing the elapsedtime between successive vehicle location points as determined by signalsfrom the GPS receiver (307) may permit calculation of average vehiclevelocity between those points. As also discussed above, accelerometer(317) of FIG. 3 may be used to measure acceleration of the motor vehicleincluding dangerous or abnormal acceleration. Such information may forma further basis for the issuance of driver warning signals from thedevice control unit of FIG. 3.

External cameras, radar, and/or lidar (light detection and ranging)sensors mounted on the motor vehicle may also be used to detectdangerous driving conditions including situations when the motor vehicleis following vehicles in front of it too closely, vehicles behind themotor vehicle are following too closely, or motor vehicles being presentin the “blind spot” on the drivers or passenger side of a moving motorvehicle. Detection of such dangerous driving situations may generatedriver warnings and be used as input to the artificial intelligenceexpert system analysis systems and methods of this invention.

Control is then passed to the vehicle motion decision block (1104) asshown in FIG. 11A. If no vehicle motion has been detected, control isreturned via the indicated control path to continue processing of motiondetection signals in block (1103) of FIG. 11. This ensures that if themotor vehicle is not moving, all telecommunication devices and cellularphones that may be present in the motor vehicle may be used by thevehicle driver or other passengers in the vehicle. If it is determinedin the vehicle motion decision block (1104) that indeed the vehicle ismoving, control is passed to processing block (1106) to further evaluatedriving conditions. Block (1106) evaluates detected vehicle motionincluding vehicle velocity or speed and erratic driving patternsincluding dangerous swerving, excessive lane changing, abnormalacceleration, driver driving history, and the like. Road conditions forthe route of travel may also be evaluated at this point includingparameters reflecting traffic and/or pedestrian congestion, accidents,weather conditions, road conditions including such considerations assurface problems, roadway width, number of lanes, erratic drivers foundin proximity to the vehicle of interest, and the like. Motor vehicleoperational parameters such as tire pressure, engine heat, brakeconditions, oil pressure, fuel levels, vehicle age, vehicle safetyinspection check and sticker status, vehicle registration status, and/orother such motor vehicle safety parameter information may also beevaluated in block (1106) of FIG. 11A. Selected information forevaluation in block (1106) may be obtained from the vehicle sensors asindicated in FIGS. 2 and 3 and/or from the motor vehicle telematicsprocessor and data base storage unit (1105) of FIG. 11A.

As further shown in FIG. 11A, decision element (1107) makes use of theresults from the evaluation of the driving and motor vehicle conditionsat (1106) to decide whether or not a dangerous driving situation thatneeds addressing is present. If no such dangerous situation exists,control is returned via the indicated path to continue processing ofmotion detection signals at processing block (1103). For example, it maybe the case of the motor vehicle is moving at a relatively slow speed ina perfectly safe driving environment such as a very straightwell-maintained road way with little or no traffic congestion or othereminent danger to the motor vehicle and its passengers. It may also bethe case that the motor vehicle is traveling at moderate speeds and thedriving conditions and history of the driving habits of the driver didnot warrant any alarm that there is a dangerous situation. On the otherhand, if the danger decision element (1107) determines that apotentially dangerous situation may exist, control is passed toprocessing block (1108) to further rank driving conditions as determinedfrom the previous processing steps of FIG. 11A. In this way, the devicecontrol unit may more accurately respond to situations that arepotentially more dangerous than others.

In addition to evaluating and taking into consideration motion of themotor vehicle, the processing block (1108) may also make use ofartificial intelligence expert systems (312) of FIG. 3 to includeconsideration of dangerous driving situations arising from traffic andor pedestrian congestion, weather conditions, and road way situationsincluding dangerous roadways being traveled and emergency trafficsituations as discussed above. More detailed discussion of suchartificial intelligence expert system technology that may be used in thesystems and methods of the present invention are provided belowincluding the use of fuzzy logic decision-making (313) which may beincluded in the implementation of the device control unit (300) asindicated in FIG. 3.

Having performed such ranking of dangerous driving conditions, controlis passed via connector “B” (1109) to the further processing and controlsteps of FIG. 11B. FIG. 11B is a continuation from FIG. 11A viaconnector “B” (1109). Having determined that a dangerous situation mayexist based on the motor vehicle motions and driving conditions, thenext step is to determine whether or not a telecommunications device orcellular telephone is being used in the motor vehicle or otherpotentially dangerous distractions for the driver exist. As indicated inFIG. 11B, multiple sensors including, for example, near fieldcommunication (NFC) signal sensors (1111), directional RF signal sensors(1112), directional array microphone sensors (1113) and camera imagesensors (1114) may be used alone and/or in combination with othersensors to determine whether or not the driver of the motor vehicle isusing a wireless device such as a cellular telephone or is otherwisedistracted from proper operation of the motor vehicle. These varioussensors are connected via software and electrical and/or opticalpathways (1110) to motor vehicle telematics processor display anddatabase storage unit (1106) and/or the device control unit designatedas (1124). These connections enable further processing and storage ofsensor information as well as displaying particular parameters ofconcern or importance to the driver via display units of the telematicsprocessor.

As discussed above and illustrated in FIGS. 3, 4 and 5, near fieldcommunication (NFC) sensors (1111) may be used to detect the presence ofNFC signals and determine the location of emanation of those signalswithin a motor vehicle. As shown in FIG. 11B, at block (1111) thatdetection is used with the other sensors described as above to determinewhether or not the driver of the motor vehicle is using a wirelessdevice such as a cellular telephone.

Directional RF signals from the area occupied by the driver of the motorvehicle are detected in block (1112). This determination may be madeusing the directional RF antenna and receiver (808) of FIG. 8. Asillustrated in FIG. 8, the directional antenna and receiver (808) ismost sensitive in a restricted beam area (807) that primarily covers thearea surrounding the driver (802) of the motor vehicle. Signals fromtelecommunications device or cellular telephone (805) being used bypassenger (803) are outside the primary lobe or coverage area of thedirectional antenna and receiver (808). Signals (806) from thetelecommunications device or cellular telephone (804) being used by themotor vehicle driver (802) will be received as stronger signals comparedto others that are outside of the main coverage area lobe as illustratedin FIG. 8. In addition, the directional antenna and receiver (808) maybe mounted to the rear of driver (802) to minimize picking up RF signalsfrom other telecommunication devices or cellular telephones that may belocated in the rear of the motor vehicle, for example, being used bypassengers in rear seats of that vehicle. Threshold levels for the RFsignals may be set to facilitate distinguishing stronger signals fromthe driver (802) from those that might be emanated from cell phones usedby passengers such as cell phone (805) from passenger (803) of FIG. 8.Dynamic adjustment of such threshold levels may be made based on“learning algorithms” implemented in the RF receiver (808) and/or devicecontrol unit (102/300). Such “learning algorithms” may be integratedwith other sensors to assist in setting threshold levels when thoseother sensors also indicate that the driver is using his or her cellulartelephone or other wireless device.

In the United States two different cellular telephone modulationstandards have emerged and are in use—GSM and CDMA. Both GSM and CDMAoperate in the UHF (Ultra-High Frequency) bands. GSM (Global System forMobile communications) makes use of frequency division multiplexing with8 or 16 timeslots defined and shared by multiple calls on each of thefrequencies. GSM is widely adopted throughout the world. The GSMAssociation estimated that in 2010 the GSM standard served 80% of theglobal mobile market or more than 5 billion people in more than 212countries making it by far the most widely used mobile communicationnetwork standard.

CDMA (Code Division Multiple Access) makes use of a different modulationtechnique than GSM. CDMA is based on spread spectrum technology withdigital signals for individual calls occupying the same frequency rangesand being separable based on unique digital coding for individual calledchannels. In the United States CDMA operates in the 800-MHz and 1900-MHzfrequency bands.

Regardless of whether GSM or CDMA technology is used, the directionalantenna and receiver (808) need only detect the presence of an RF signalin the respective frequency ranges to verify the operation of atelecommunications device or cellular telephone being used by the driverof the motor vehicle as illustrated in FIG. 8. Multiple techniques forimplementing such a receiver are known to those of skill in the art ofradio receiver design. Simpler designs that only detect the presence ofsuch RF signals may be used, or more complete GSM and CDMA receiver suchas used in cellular telephones and capable of the demodulatingindividual channels may also be used.

Block (1113) processes directional array microphone signals asillustrated for example in FIG. 6 and explained above relative to thatfigure. A directional microphone array (605) may use acousticbeamforming and other noise reduction algorithms as explained above toisolate speech from the driver of the vehicle and those of otherpassengers, wind noise, road noise, motor vehicle engine noise or otherinterfering acoustic signals.

Block (1114) provides for processing of captured camera image signals asillustrated in FIG. 10 and discussed above. Such image analysis mayprovide useful information concerning activities of the driver and inparticular the driver's use of cellular telephones or other wirelessdevices for communication purposes.

Based on the results of the above described sensor analysis, control ispassed to decision element (1115) to direct further activities dependingon the results of that analysis. Such analysis may be based on outputsfrom the collection of sensor analysis blocks (1111), (1112), (1113) and(1114) to provide evidence of dangerous driver distractions arising fromthe use of cellular telephones or other wireless devices. The outputsfrom some of the above sensors may be inconclusive while others may givea positive indication of driver use of these telecommunication devices.Certainly, if all sensors indicate such use, a decision is madeconfirming that use. Other combinations of results may be evaluated withdecisions made based on individual sensor outputs and the combination ofthose outputs. For example, the output of the directional RF signaldetection may be inconclusive but the near field communication sensorssignals and microphone array signals may indeed indicate that the driveris involved in the use of cellular telephones or other wireless devices.The application of artificial intelligence expert system technology andfuzzy logic to further assist in evaluating dangerous driverdistractions in combination with other dangerous driving conditions isdiscussed further below.

If no driver telecommunication device or cellular telephone signals aredetected, control is passed to decision element (1116) for determinationof whether or not other cognitive distractions in the form of audiosignals from passenger conversations or other sources are present. Suchadditional audio cognitive distractions may be detected, for example,using omnidirectional antenna and associated receiver (324) of FIG. 3and/or results from signals received by the directional microphonearrays (303) of FIG. 3 and further illustrated as directional microphonearrays (605) and (606) of FIG. 6. If such additional cognitivedistractions are detected control is passed via connector “C” (1123) forissuance of appropriate warnings and/or file history entries.

Even if no other cognitive disruptive audio is detected at decisionelement (1116), control is still passed to connector “C” of FIG. 11B asdescribed above with notification that no telecommunication device orcellular telephone is being used and no other cognitive distractionshave been detected. However, based on other dangerous driving conditionsdetected as indicated in FIG. 11A and discussed above, warning signalsmay still be issued to the driver.

If the wireless device usage decision element (1115) indicates that thedevice control unit (300) has determined that a dangerous drivingsituation exists and based on the sensor analysis (1111), (1112), (1113)and/or (1114) that positive sensor results indicate a driver is usinghis or her wireless device while driving in a dangerous situation,control may next be passed to the speech-to-text conversion block (1117)of FIG. 11B. Using speech-to-text conversion with accompanyingrecognition of selected words or phrases will provide further indicationthat the driver of the moving motor vehicle is talking on atelecommunication device or cellular telephone as opposed to texting orusing the cellular telephone or wireless device in some other manner.Control is next passed to speech detection decision block (1118). Basedon the above described sensor and speech to text conversion analysis adecision is made as to whether or not actual driver speech has beendetected. If no speech has been detected, but indication exists from theabove described sensor and signal analysis that the driver is using awireless device or cell phone and driving in a dangerous situation,control is passed to further process the received RF signals at block(1121).

The decision as to whether or not the driver may be texting on his orher wireless device or cellular telephone is made in decision element(1122). Such determination can be made based on analysis of the RFsignal with demodulation of the any actual data signals. If no signalscorresponding to actual text transmissions can be detected, and controlis passed to connector “C” (1123) for registering final decisions atinformation block (1124) as indicated in FIG. 11B. At this point theactual testing has been indeterminate in ascertaining whether or not thedriver is talking or texting using a wireless device or cell phone whiledriving in a dangerous situation. Nonetheless, at least dangerousdriving conditions have been detected and appropriate warnings to thedriver may be displayed at block (1124). Control is then passed viaconnector A (1101) to resume monitoring for dangerous driving situationsas depicted in FIG. 11A. If the texting decision element (1122)concludes of the driver is indeed texting, control is passed viaconnector “C” (1123) to reflect the situation in the accumulatedinformation and to permit ranking of driver distraction caused by suchtexting using artificial intelligence expert system analysis asdescribed below. Based on the results of this analysis, appropriatewarnings may be displayed by unit (1124) of FIG. 11B.

On the other hand, if decision element (1118) determines that actualspeech has been detected, control is then passed to decision element(1120) to determine whether or not the detected speech has resulted froma cellular telephone or other wireless device being held to the user'sear for the purpose of communication while driving in a dangeroussituation. This decision may be made, for example, based on the resultsof near field communication sensors analysis (1111) and/or of the cameraimage signals at block (1114). If the answer to this question is “yes,”control is passed to connector “C” (1123). Connector C passes control tothe rank driver distraction block (1124) discussed below as shown inFIG. 11B.

If it is determined at hand held decision element (1120) that thecellular telephone or wireless communication device is not being held bythe user, the control is passed to the hands-free speech decisionelement (1119). The hands-free speech detection decision (1119) may alsoaccess the motor vehicle telematics processor for an indication that ahands-free telecommunications call is indeed in progress. Based on thatinformation and the information from the above described NFC sensors(1111), directional RF signal sensors (1112), directional arraymicrophone signal sensors (1113) and/or camera image signal sensors(1114) it can be determined whether or not the hands-free communicationcall involves the driver of the motor vehicle. If it is determined thatthe cell phone or other wireless communication device is being used in ahands-free mode, control is passed to connector “C” (1123). Connector Cpasses control to the rank driver distraction block (1124) discussedbelow as shown in FIG. 11B.

The ranking of driver distraction and dangerous driving conditions inblock (1124) then provides an indication based on the motor vehiclemotion, motor vehicle conditions, roadway conditions, traffic and/orpedestrian congestion, emergency reporting situations, drivingdistractions including those arising from the use of cellular telephonesor wireless devices as described above, or other driving distractionscaused by other activities in a moving vehicle. As discussed above, allof this information is available in the motor vehicle database forevaluation of the overall degree of danger and for determination ofappropriate warnings and/or control signals. This evaluation and rankingof a dangerous situation may be carried out using artificialintelligence expert system methods as described below. Such artificialintelligence may include the use of fuzzy logic reasoning as alsodescribed below.

An appropriate warning signal may be displayed or action may be taken toterminate the communication from the wireless device being used by thedriver of the motor vehicle. Information recording driver violation ofsafe driving practices may be recorded in a history file in the databaseof device control unit (1124) shown in FIG. 11B. For example, thewarning signals or commands may be issued via speaker (318) or bymessages displayed on display unit (320) of FIG. 3 or by other emergencywarning implementations designed to be certain the driver and passengersin the motor vehicle are aware of the danger and appropriate correctionsto ensure their safety are made.

In one embodiment of the present invention, the device control unit(102/300) of FIGS. 1, 2 and 3 may issue a command to thetelecommunications device or cellular telephone to inhibit itsoperation. Such commands may be transmitted, for example, via cellulartransceiver (308), data transceiver (309), Bluetooth transceiver (310)or Wi-Fi transceiver (311) of FIG. 3 used as discussed above andconfigured to transmit such control commands to the telecommunicationsdevice or cellular telephone being used by the driver of the motorvehicle. One technology useful in transmitting such messages are SMScontrol messages that may be “pushed” to the telecommunications deviceor cellular telephone without first being requested by that device ortelephone and which may be used with application software to inhibit theuse of the telecommunications device or cellular telephone that is beingused in a dangerous manner by the driver of the moving vehicle.

As indicated in FIG. 11B and discussed above, wireless decision usageelement (1115) depends upon the outputs from the four exemplarypassenger activity sensors (1111), (1112), (1113) and (1114). These fourexemplary sensors may include near field communication (NFC) sensors,directional RF signals, the directional array microphone signals andcamera image signals. Based on the outputs of the sensors, the wirelessdevice usage decision element (1115) decides whether or not evidencefrom the sensors indicates with some degree of certainty that the driverof the vehicle is involved in dangerous usage of a cellular telephone orwireless device. If such usage is confirmed, control is passed toelements (1117) through (1124) for further signal analysis. In general,no one of the sensors by themselves may be considered to confirmdangerous usage by the driver of a cellular telephone or wirelessdevice. For example, false alarms are possible. The near fieldcommunication signal (NFC) sensor may be responsive to a device notbeing used by the driver. The RF signal sensor may be responsive to asignal not emanating from a device being used by the driver. Themicrophone array signal sensors may be responsive to speech or audiosignals not being generated by the driver of the motor vehicle. And thecamera image signal analysis may respond to images that did notnecessarily correspond to cellular telephone or wireless device usage bythe driver of the motor vehicle. These issues of determining with anappropriate level of certainty that the driver of the motor vehicle isinvolved in dangerous usage of cellular telephones or wireless devicesis further complicated by the presence of other passengers in the motorvehicle who may be using such devices or otherwise generating signals towhich the above-described sensors may be responsive.

Without limitation, FIG. 12 presents an expert sensor decision matrix(1200) of the type that may be used by wireless device usage decisionelement (1115) of FIG. 11B. This matrix (1200) sets forth possible cellphone-wireless device usage sensor decisions for each of the 16 possiblecombinations of results from the above for identified driver activitysensors (1111), (1112), (1113) and (1114) as discussed above. In thismatrix, a “0” indicates a negative result from the sensor and an “X”indicates a positive result. The decision matrix of FIG. 12 requiresthat three of the four sensors indicate positive results for dangerousdriver activity involving the use of a cellular telephone or otherwireless device. As shown in FIG. 12, 5 of the 16 possible combinationsof sensor results indicate dangerous usage of a cellular telephone orwireless device by the driver of the motor vehicle. Of course, it ispossible that other expert decision criteria may be used to make thefinal decision in the wireless device usage decision element (1115) ofFIG. 11B without departing from the teachings and scope of the presentinvention. For example, fewer than three out of four sensor positiveresults may be sufficient to indicate that the driver is involved indangerous usage of a wireless device or cellular telephone. In addition,more weight may be given to the output of some sensors than othersensors. In addition, the outputs of individual sensors may be graded orranked based on the strength of the indication from those individualsensors. Artificial intelligence expert systems and/or fuzzy logic maybe used to assist in making such decisions. The use of such artificialintelligence expert system technology may prove useful in someembodiments to be used to supplement or in place of the yes/no discretedecisions depicted in FIG. 12.

Artificial Intelligence Expert System and Methods Description

Artificial intelligence expert system technology may be used to assistin determination of appropriate warning and control signals at block(1124) of FIG. 11B. Without limitation, an exemplary artificialintelligence expert system implementation involving four criticalvariables in developing a comprehensive driver danger-warning index isdescribed below. The implementation presented here is meant to bedescriptive of such a comprehensive system but is not intended to limitin anyway the application of artificial intelligence expert systemtechnologies and/or fuzzy logic to such a system. Different variables indifferent combinations may be defined based on the principles set forthherein and remain within the scope of the invention herein described.The variables considered here are: (1) roadway conditions, (2) weatherconditions, (3) traffic conditions and (4) driver distraction. Each ofthese four variables depend upon multiple considerations that may alsobe included in implementation of this invention described herein withoutdeparting from or adding to the teachings herein presented.

Without limitation, FIG. 13 depicts an artificial intelligence fuzzylogic control (1300) system for deriving driver danger warning signalsuseful in some embodiments of the present invention. The controller(1300) comprises an artificial intelligence fuzzy logic controller(1302) used for analyzing variable signal inputs (1301) as indicated inthe figure. Those signal inputs include (1) “r” roadway conditions, (2)“w” weather conditions, (3) “t” traffic conditions and (4) “d” driverdistraction signals. The input signals are processed by the controller(1302) producing output warning and control signals (1303) as describedmore completely below.

Without limitation, FIG. 14 depicts a flowchart (1400) useful theimplementation of the processing unit (1124) of FIG. 11B. The flow chartdepicts possible artificial intelligence expert system processing of thefour roadway condition, weather condition, traffic condition and driverdistraction signals (1401) depicted in FIG. 13. Processing begins atblock (1402) with input of the input roadway and weather conditionsensor network data inputs. A warning index (1403) based on these inputsis computed at block (1402). Input traffic congestion data is input atblock (1404). This input traffic congestion data together with theroad/weather warning index computed in block (1402) is used to compute adriving warning index (1405) based on all three of theroad/weather/traffic sensor network data inputs. At block (1406) inputdata characterizing motor vehicle driver and/or passenger activitiesthat may represent distractions to the driver of the motor vehicle areinput. This driver distraction data is used at block (1406) to compute acomprehensive driver warning index (1407) based on all four inputvariables—roadway conditions, weather conditions, traffic conditions anddriver distractions. More detailed descriptions of exemplary artificialintelligence expert system implementation of the computations of FIG. 14according to the present invention are presented in FIGS. 15A to 15G anddiscussed in detail below.

While FIG. 14 describes successive considerations of the four exemplaryvariables, other artificial intelligence fuzzy logic implementationsembodiments may be used without departing from the teachings of thepresent invention. For example, embodiments that consider differentcombinations of variables or all of the variables in a single,multi-dimensional, fuzzy logic calculations are possible. While suchalternatives are possible and within the scope of the present invention,the approach of FIG. 14 provides information in output signalsreflecting where the real dangers exist and potentially better informingthe driver of the nature of actual dangerous situations.

FIG. 15A presents an artificial intelligence expert system decisionmatrix (1500) involving the roadway condition and weather conditionvariables. This matrix may be used to derive a road/weather warningindex based on the degree of danger to the driver and passengers of amotor vehicle arising from roadway and weather conditions. As describedfurther below, the resulting road/weather warning index may be used withtraffic and driver distraction third and fourth variables to derive anoverall artificial intelligence expert systems assessment of danger tothe driver and passengers of the motor vehicle.

Considering first the decision matrix (1500) depicted in FIG. 15A, theroadway condition variable may be categorized as corresponding to: (1)very low danger, (2) low danger, (3) medium danger, (4) high danger or(5) very high danger. For example, the roadway degree of danger may bedetermined from such considerations such as roadway surface conditions;slippery roadway, roadway construction projects; roadway width; roadwayincline; roadway location including, for example, a switchback ormountainous roadway involving steep drop-offs; roadway signage; thepresence or absence of roadway control signaling such as stoplights orwarning lights or warning signs; the number of traffic lanes; railroadcrossings; crossroads; and/or roadway accident history or other similarvariables that might impact safety considerations with respect to theroadway being traveled. Considering these various variables, a roadwayexpert safety engineer or equivalent personnel may designate the dangerparameters for the given roadway into one of the above five categories.

For example, the following roadway conditions may be defined:

-   -   roadway danger very low—very little or no roadway issues with no        known issues arriving from roadway surface conditions, dangerous        driving conditions, warning or signal issues, crossroads,        accident history or the like,    -   roadway danger low—minor roadway issues and no known issues        arising from accidents, high-speed chases, police activities or        the like,    -   roadway danger medium—moderate roadway issues with at least one        issue involving accidents, high-speed chases, police activities        or the like,    -   roadway danger high—more serious roadway issues with        complicating traffic issues such as accidents, high-speed        chases, police activities or the like,    -   roadway is danger very high—bad roadway issues with very high        traffic congestion levels and serious complicating traffic        issues involving accidents, high-speed chases or police        activities or the like.

In a similar way, FIG. 15A is based on the use of a variable describingweather conditions on the roadway being traveled. Weather conditions mayinclude considerations such as the following: no weather issueswhatsoever, temperature, rain of varying intensity, hail of varyingintensity, snow of varying intensity, ice and/or snow on the roadway,high winds, tornadoes, hurricanes, blizzards and the like. Here again aroadway weather expert safety engineer or equivalent personnel maydesignate the danger parameter for the given conditions into one of theabove five categories.

For example, the following weather conditions may be defined:

-   -   weather danger very low—very little or no roadway current or        forecast weather issues or the like,    -   weather danger low—minor weather issues with light winds and        light rain sprinkles or the like,    -   weather danger medium—moderate weather issues with heavier rain        or light snow or the like,    -   weather danger high—more serious weather issues with heavy rain        or hail or snow or the like,    -   weather danger very high—very serious weather issues with        accumulating snow, ice, flooding, high winds of the like.

Based on the combination of roadway danger and weather conditions asdescribed above, a road/weather warning index may be derived from thematrix of FIG. 15A. For example, as indicated in the figure, if theroadway poses high danger and the weather poses medium danger then theroad/weather index will be “high.” The proper assignment of road/weatherwarning indices in FIG. 15A may be provided by expert traffic engineersbased on experience and safety/accident records under the describedconditions.

The 25 results for the road/weather warning index values shown in thematrix of FIG. 15A may be described in terms of propositional calculusformulations, for example, as follows:

-   -   If the road danger is very low and the weather danger is very        high, then the road/weather warning index is high.    -   If the road danger is high and the weather danger is low, then        the road/traffic warning index is high.    -   If the road danger is very high and the weather danger is low,        then the road/traffic warning index is high.

Clearly 25 such propositional calculus statements exist for theroad/weather warning index matrix (1500) of FIG. 15A. The weather androadway danger indices of FIG. 15A are derived from external systems asindicated, for example, in FIG. 2. The parameters for the road/weatherwarning index matrix of FIG. 15A are transmitted via the motor vehiclecommunication system (204) to the motor vehicle telematics database andprograms (201) and device control unit (102/300) of FIGS. 2 and 3 forfurther integration into and use in the artificial intelligence expertsystem analysis of the present invention.

The intelligent system matrix of FIG. 15A and the associatedpropositional logic expressions may also be used to formulate a fuzzylogic implementation of the device control unit (102/300) of the presentinvention. Fuzzy logic has found expanded uses in the development ofsophisticated control systems. With this technology complex requirementsmay be implemented in amazingly simple, easily managed and inexpensivecontrollers. It is a relatively simple method of representing analogprocesses on a digital computer. It has been successfully applied in amyriad of applications such as flight control systems, camera systems,antilock brakes systems, wash machines, elevator controllers, hot-waterheaters, and stock trading programs.

In the present invention, the variable ranges for roadway danger andweather danger indicated in FIG. 15A may be “fuzzified” as fuzzy logicvariables extending over the defined overlapping ranges as shown, forexample, in the fuzzification diagrams 1501 of FIG. 15B. With fuzzylogic control, statements are written in the form of the propositionallogic statements as illustrated above. These statements representsomewhat imprecise ideas reflecting the states of the variables. Fuzzylogic is particularly appropriate when an expert is available to specifythese propositional statements characterizing the relationships betweensystem variables.

Fuzzy logic systems make use of “fuzzifers” that convert input variablesinto their fuzzy representations. “Defuzzifiers” convert the output ofthe fuzzy logic process into “crisp” numerical values that may be usedin system control.

For example, the graph (1502) of FIG. 15B illustrates such a possible“fuzzification” for the roadway danger index variable of FIG. 15A withoverlapping ranges indicated in the figure. Numerical values from 0 to10 may be assigned to the respective input roadway and weather variablesdepending on their classification as low, very low, medium high or veryhigh as described above. For example, values of 1, 3, 5, 7 or 9 may beassigned to the respective classifications, as well as values betweenthese values depending on the expert programming of the system.

In this example, based on the combination of roadway conditions derivedfrom the sensor networks and other inputs, a roadway danger index of 5.7has been determined. As indicated in the diagram of (1502), the fuzzylogic calculations depicted here are based on triangular degree ofmembership (DOM) classifications corresponding to the very low, low,medium, high and very high parameter values indicated in the matrix ofFIG. 15B. Plotting the roadway danger index of 5.7 on the horizontalaxis indicates a degree of membership in the medium classification of0.67 with no membership for this variable in any of the other indicatedtriangular ranges. Only the “medium” membership function “fires.” Evenso, the DOM of 0.67 indicates a less than maximum membership in the“medium” range.

In a similar way, in the diagram of (1503), the indicated whether dangerindex of 7.0 results in a degree of membership (DOM) of 0.4 in thetriangular high danger classification and 0.2 in the triangular mediumdanger classification. Both the high and medium membership functions are“fired” reflecting the “fuzzy” nature of the weather input data.

These DOM values may in turn be used in the fuzzy logic implementationto derive a defined, “crisp” numerical value for the combinedroadway/weather warning index. This process is called “defuzzification.”Without limitation, in one embodiment of the present invention,defuzzification may be based on a logical “or” relationship betweenrespective DOM values for the variables to be defuzzified.

The conjunctive relation “OR” corresponds to the logical union of thetwo sets corresponding to the roadway danger and weather danger indexvariables. In this case the appropriate DOM is the maximum DOM for eachof the sets at the specified time. This is expressed algebraically asfollows:(A∪B)(x)=max(A(x),B(x)) for all x∈X

Without limitation, in other embodiments of the present inventiondefuzzification may be based on a DOM logical “and” relationship.Premises connected by an “AND” relationship are combined by taking theminimum DOM for the intersection values. This is expressed algebraicallyas follows:(A∩B)(x)=min(A(x),B(x)) for all x∈X

Consider the logical “or” relationship as described above. The unionrelation “OR” requires the use of the maximum value of the respectiveDOM's for roadway danger and weather danger. Use of the maximum DOMfocuses attention on the more dangerous parameter values.

From the graphs (1502) and (1503), for these propositional logicequations the corresponding DOM's are 0.67 for the roadway dangervariable medium DOM and 0.4 for the weather danger variable high DOM.

These values may be used to defuzzify low and medium ranges of theroadway and weather warning action indices degree of memberships. Asshown in (1504) of FIG. 15B, fuzzy ranges for the combinedroadway/weather index may be defined in a similar manner to the roadwayand weather variables. A numerical “crisp” value for the combinedroadway/weather warning index can now be derived using defuzzificationprocedures. As shown in FIG. 15B, the DOM ranges for the roadway/weatherwarning index are capped at values corresponding to the above analysisfor the DOMs of the roadway danger and weather danger variables. Withoutlimitation, the final “crisp” numerical value of the driver warningaction index may be calculated based on the centroid of the geometricfigure for the low and medium DOM ranges of the graph (1504) of FIG.15B. For example, this calculation may be carried out by dividing thegeometric figure of FIG. 1504 into sub-areas A, each with knownindividual centroids x_(i) from the following formula.

$x_{C} = {\left( {\sum\limits_{i = 1}^{n}{x_{i}A_{i}}} \right)/\left( {\sum\limits_{i = 1}^{n}A_{i}} \right)}$

The result of such a calculation is shown in FIG. 15B yielding aroadway/weather warning action index crisp numerical value of about6.75. Note that this value is somewhat less than the weather index of 7and somewhat more than the roadway index of 5.7. The value of 6.75reflects the greater concern for weather over roadway dangers.

As described above, in some embodiments of this invention,multi-dimensional fuzzy logic calculations based on multiple inputvariables with a resulting multi-dimensional surface for defuzzificationmay also be used. In this case, more than two individual input variablesmay be fuzzified simultaneously in the same manner as shown in FIG. 15Band discussed above. Multiple degree of membership results may then beused to result in a multidimensional surface calculation fordefuzzification and derivation of a crisp output value.

In a similar way, the artificial intelligence expert system decisionmatrix (1505) of FIG. 15C is constructed using a combination of theroad/weather warning index derived above with a third variableindicative of danger level of traffic on the roadway being traveled. The25 entries in the matrix (1505) of FIG. 15C set forth a compositemeasure of vehicle driving danger due to roadway conditions, weather andtraffic. The traffic danger variable of this matrix is categorized ascorresponding to: (1) very low danger, (2) low danger, (3) mediumdanger, (4) high danger or (5) very high danger. For example, the degreeof danger arising from traffic considerations may be determined fromconsiderations involving the volume of traffic on the road, accidents onthe roadway, traffic delays, known erratic drivers on the roadway, highspeed chases, police activities, pedestrian traffic density and thelike. Once again, considering these various variables a traffic expertsafety engineer or equivalent personnel may designate danger parametersfor the giving traffic conditions into one of the above five categories.

For example, the traffic danger variable may be categorized into one ofthe above five degrees of danger in accordance with considerations suchas those exemplified below:

-   -   traffic danger very low—very little or no traffic with no known        issues arriving from accidents, high-speed chases police        activities or the like,    -   traffic danger low—low traffic levels, no known issues arriving        from accidents, high-speed chases, police activities or the        like,    -   traffic danger medium—moderate traffic levels with at least one        issue involving accidents, high-speed chases, police activities        or the like,    -   traffic danger high—high traffic levels with complicating        traffic issues such as accidents, high-speed chases, police        activities or the like,    -   traffic is danger very high—very high traffic congestion levels        with serious complicating traffic issues involving accidents,        high-speed chases or police activities or the like.

Here again, the 25 results for the road/traffic warning index valuesshown in the matrix of FIG. 115C may be described in terms ofpropositional calculus formulations, for example, as follows:

-   -   If the traffic danger is very low and the road/weather danger is        very high, then the driving warning index is high.    -   If the traffic danger is high and the road/weather danger is        low, then the driving warning index is medium.    -   If the traffic danger is very high and the road/weather danger        is low, then the driving warning index is high.

Once again, 25 such propositional calculus statements exist for thevehicle driving warning index matrix (1505) of FIG. 15C. The trafficcongestion and road/weather danger indices of FIG. 15C are derived fromexternal systems such as the roadway/traffic monitor (207) of FIG. 2.The parameters for the vehicle driving warning index matrix of FIG. 15Care transmitted via the motor vehicle communication system (204) to themotor vehicle telematics database and programs (201) and device controlunit (102) of FIG. 2 for integration into and use in the artificialintelligence expert system analysis of the present invention.

FIG. 15D depicts a fuzzy logic implementation (1506) combining theresults of the road-weather index “crisp” value of 6.75 derived in FIG.15B (1504) with a traffic danger index of 4.0 to compute a “crisp” valuefor a combined road-weather-traffic warning index. The calculationoutlined in FIG. 15D is based on the logical “or” combination of thesevariables as discussed above. The road-weather “crisp” warning index of6.75 from FIG. 15B results in firing of both the medium and high DOMfunctions for that index as illustrated in (1507). The traffic dangerindex of 4.0 fires only the low DOM function as illustrated in (1508).Using the maximum values and the centroid calculation proceduredescribed above results in a composite road-weather-traffic warningindex of 5.65 as shown in FIG. 1509). Note that this value is somewhatless than the road-weather index of 6.75 reflecting the indicated lowtraffic danger value.

FIG. 15E presents an artificial intelligence expert system decisionmatrix (1510) for a driver warning index based on the derivedroad-weather-traffic warning index of FIG. 15D and a derived driverdistraction index based upon the above described analysis of FIGS. 11A,11B and 12. Here again the derived driver distraction index iscategorized as corresponding to: (1) very low danger, (2) low danger,(3) medium danger, (4) high danger or (5) very high danger. For example,the degree of danger arising from the driver distraction will dependupon analysis of sensor outputs and derived variables as describedabove. Once again, considering these various variables a driverdistraction expert safety engineer or equivalent personnel may designatedanger parameters for the given driver distraction situations into oneof the above five categories. As described in the above analysis forFIGS. 11A, 11B and 12 and also in FIGS. 1-10, the driver distractionindex will be placed in one of the above five categories based on driverdistractions arising from the use of hand-held cellular telephones orother wireless devices, the use of cellular telephones or other wirelessdevices for texting, and other distractions of the driver arising, forexample, from conversations between the driver and other passengers ofthe motor vehicle or between passengers of the motor vehicle and/orother audio or visual distractions arising in the vehicle. Visualdistractions may arise for example from operation of automotiveTelematics display units for such purposes as navigation, entertainmentcontrol, and display control to learn of motor vehicle conditions,weather conditions, or other information. Modern motor vehicletelematics systems also present distractions for the control of motorvehicle internal environmental such as heat or cooling or fan levels.All of these various parameters may be considered by the above describeddriver distraction expert safety engineer or equivalent personnel indetermining the level of driver distraction as indicated in FIG. 15E.

For example, without limitation, the driver distraction variable may beassociated with driver activities as follows:

-   -   driver distraction very low—no driver use of telecommunication        device or cellular telephone and quiet conditions in the moving        motor vehicle,    -   driver distraction low—driver involved in conversations with        other passengers or others in the motor vehicle talking on        separate telecommunication devices or cellular telephones or        involved in distracting conversations    -   driver distraction medium—driver talking using hands free        connection on telecommunication device or cellular telephone,    -   driver distraction high—driver talking on handheld        telecommunication device or cellular telephone,    -   driver distraction very high—driver texting on        telecommunications device or cellular telephone.

FIG. 15F depicts a fuzzy logic implementation (1511) combining theresults of the vehicle driving warning index “crisp” value of 5.65derived in FIG. 15D (1509) with a driver distraction index of 7.0 tocompute a “crisp” value for a composite driver warning index. Thecalculation outlined in FIG. 15F is again based on the logical “or”combination of these variables as discussed above. The vehicle drivingdanger index of 5.65 fires only the medium DOM function as shown in FIG.15F (1512). The driver distraction index of 7.0 results in firing ofboth the medium and high DOM functions for that index as shown in FIG.15F (1513). Using the maximum values and the centroid calculationprocedure described above results in a composite driver warning index of6.4 as shown in FIG. FIG. 15F (1514). Note that this value is somewhatless than the driver distraction danger index of 7.0 reflecting theindicated lower vehicle driving danger value of 5.65.

FIG. 15G illustrates the calculation (1515) of FIG. 15F but with thedriver distraction index changed from 7.0 to 3.0 as illustrated in(1517). The vehicle driver danger index calculation (1516) is the sameas in (1512) of FIG. 15F. As indicated in this case the driverdistraction index fires the low DOM but none of the other driverdistraction index DOMs. In this case the driver distraction index DOMmembership in the low range is equal to 0.85. This lower DOM indicates aconsiderable decrease in the driver distraction from the value of 7.0used in FIG. 15F. Using the same calculation procedure as used for FIG.15F with defuzzification based on the centroid calculation methodresults in a composite driver warning index of 3.7 as indicated in thediagram (1518) of FIG. 15G. In this case the overall driving warningindex of 3.7 is larger than the driver distraction index of 3.0. Thisreflects the fact that the vehicle driver danger warning index of 5.65contributes to the overall danger of the driving situation resultingfrom roadway, weather and traffic considerations. Unlike the calculationof FIG. 15F, in this case the composite driver warning index is greaterthan the driver distraction index compared to the results of FIG. 15Fwhere the composite driver warning index of 6.4, is less than the driverdistraction index of 7.0. Comparison of the results of FIGS. 15G and 15Fdemonstrate the influence of the roadway, weather and traffic dangers onthe overall composite driver warning index of the present invention.

The artificial intelligence expert system warning derivation matrices ofFIGS. 15A-15G may also be stored with multiple versions correspondingto, for example, the detection of different drivers based on imageanalysis facial recognition as described above. In this regard, somedrivers may have driving habits that are more dangerous than others inwhich case the entries in the above described matrices may be changed toindicate a more dangerous situation with particular drivers. Forexample, young drivers are more likely to have accidents than otherdrivers. Teenagers are more easily distracted by activities around themincluding conversations with other passengers, listing to music or lackof concentration on the task of driving. This flexibility to adapt theartificial intelligence expert system analysis methods of this inventionmay be used to more accurately evaluate particular dangerous situations.The driving habits of particular individuals may also be learned by thesystems and methods of the present invention with corresponding updatingof the decision variables described above based on such artificialintelligence training or learning.

FIG. 15H illustrates, without limitation, the partial flow ofinformation (1519) in the expert artificial intelligence decision-makingset forth in FIGS. 15A-15G and described above. As indicated in the FIG.15H, artificial intelligence expert system calculations are executedusing the device control unit calculations as shown in block (1124) ofFIG. 11B. Database information indicating inputs from the road/weathersensor network are analyzed at block (1520). That information is in turnused for calculations of the road/weather warning index of FIG. 15A/B atblock (1521) of FIG. 15H. The results of the road/weather warning indexcalculation is combined with the input traffic sensor network data asindicated in blocks (1522) and (1523) for computing the vehicle drivingwarning index of FIG. 15C/D in block (1523) as shown in FIG. 15H. Asfurther indicated in FIG. 15H, the computed vehicle driving warningindex from block (1523) is combined with input driver/passenger activitysensor data in blocks (1524) and (1525) to compute a final compositedriver warning index as indicated in FIGS. 15F and 15G.

While the example of FIGS. 15A-15H is limited to four variables (roadwayconditions, weather conditions, traffic conditions and driverdistraction) clearly additional tables may be constructed to includeother important variables in the decision process as disclosed in thisinvention.

FIG. 16 illustrates in more detail exemplary fuzzy logic operationexecution (1600) by the device control unit (300)/(1605) for the systemand methods of this invention. As shown in the figure, these operationsinclude access to the artificial intelligence expert system knowledgebase (1604) which may include the fuzzy logic rules discussed above. Thefuzzy logic operations include the fuzzifier (1601) used to establishdegree of memberships (DOMs) as discussed above and illustrated in FIG.15A-15G. The outputs of fuzzifier (1601) are fed to the fuzzy logicprocessing element (1602). Defuzzifier (1603) provides crisp numericaloutputs for the task dispatch index as illustrated in FIGS. 15A-15G withreturn control via path (1606).

FIG. 17 illustrates, without limitation, application of the abovederived driver warning index to control a cellular telephone or otherwireless device potentially being used in a dangerous manner by thedriver of the motor vehicle. The decision process (1700) as indicated inFIG. 17 is carried out by the device control unit operations (1124) asshown in FIG. 11B. The decision process (1700) begins at block (1701)with evaluation of the above computed driver warning index. As indicatedin FIG. 15A-15G and discussed above, the output of the fuzzy logiccalculation is an analog value in the range from 0 to 10. Using thatcomputed value, control is passed to decision element (1702) to comparethat value to an expert defined threshold level to further determineappropriate actions to be taken. If the driver warning index is belowthis threshold, control is passed to block (1705) to display appropriatewarnings without locking out or inhibiting the wireless device frombeing used by the driver of the vehicle. However, if the driving warningindex exceeds the threshold as indicated by decision element (1702),then a decision must be made whether or not to lock out or inhibit useof the cellular telephone or wireless device by the driver of the motorvehicle. In some embodiments, it may be required that the threshold beexceeded for a predetermined period of time before control actions basedon exceeding the threshold are employed.

Even if the driver warning index exceeds the expert predefined thresholdlevel, extenuating circumstances may exist where it would be unwise todisable or lockout the cellular telephone or wireless device. Forexample, it may be the case that a medical emergency exists requiringimmediate attention. Such a medical emergency may in fact be detected bythe medical sensors (326) of FIG. 3 as discussed above. In such asituation it may be imperative that the driver be able to communicateover the cellular telephone or wireless device regardless of otherdanger situations that may exist. The driver may need to dial “911” orother emergency contact numbers for advice on how to respond to themedical emergency or to summon immediate assistance. Clearly in such asituation it would be a mistake to inhibit or lockout the cellulartelephone or wireless device being used by the driver. As indicated inFIG. 17, if such a medical emergency is detected control is passed againto block (1705) for displaying appropriate warnings without locking outthe wireless device.

If no medical emergency is indicated by decision element (1703), controlis passed to yet another decision element (1704) for assessment of othercritical emergencies that may exist that require the driver be able tocommunicate via his or her cellular telephone or wireless device inspite of the fact that the driver warning index exceeds the abovedescribed threshold. For example, the driver may witness or be involvedin an automobile accident, a robbery, road rage, an attack, or othersituations that require communication with outside assistance. If such acritical emergency is detected, control is again past to block (1705) todisplay appropriate warnings without locking out or inhibiting thecellular telephone or wireless communication device.

In one embodiment of this invention, all telephone calls initiated bythe driver of the vehicle by dialing “911” will be allowed to proceedregardless of any dangerous driving or driver distraction situationsdetected by the systems and methods of this invention. In otherembodiments, automatic calls for outside assistance by dialing “911” orother emergency response numbers may be made using the systems andmethods of this invention. Examples of situations where such automaticcalls may be made include, without limitation, deployment of airbags,gunshots detected by the above described microphone arrays and acousticsignal processing, signal outputs from accelerometer (317) of FIG. 3corresponding to abrupt stops or other abnormal vehicle decelerationindicative of an accident or other situations in which emergencyassistance may be required and the driver may not be able to initiaterequired communications to solicit such assistance.

Finally, if no such critical emergency is detected at decision element(1704), control is passed to the lockout wireless device use block(1706). In this case the driver warning index has exceeded the thresholdlevel and/or no emergency has been detected that would justify continueddangerous use by the driver of the cellular telephone or wirelessdevice. Control is returned again to block (1124) of FIG. 11B asdescribed and discussed above. In this way, the inventive operations andmethods of FIG. 17 preclude the inhibiting or lockout of the cellulartelephone or wireless device of the driver when emergency situationsexist that override potentially dangerous driving conditions that mayhave been detected.

In some embodiments of this invention, timers may also be used to delaycontrol signals inhibiting the use of cellular telephones or otherwireless devices by the driver of the motor vehicle. For example, insome embodiments it may be appropriate to require that the compositedriver warning index computation as indicated above to be above aspecific threshold for a specified period of time before correctiveaction is taken. In this way, spurious decisions based on transitorybehavior may be reduced or some cases eliminated. For example, if adangerous situation is indicated by the above described combination ofsensors does not persist for a specified period of time, correctiveaction may be delayed with appropriate warnings being displayed.

FIG. 18 depicts, without limitation, possible display options (1800)indicating selected of the results derived as described above in thepresent invention. For example, as explained above the calculated driverwarning index using the artificial intelligence expert system and fuzzylogic system and methods of this invention results in an analog signalbetween 0 and 10 indicating a numerical value of the driver warningindex. As shown in illustration (1802), the value of the calculatedanalog driver warning index can be displayed for easy comprehensionusing a simple analog dial graphic. As also illustrated in FIG. 18,values for the four variables used in the above described calculationsmay be indicated for easy comprehension using a simple bar-graph asdepicted in illustration (1801). The four variables (1) roadway danger,(2) weather danger, (3) traffic danger and (4) driver distraction areall simply displayed as being within the five define ranges of (1) verylow, (2) low, (3) medium, (4) high or (5) very high as illustrated in(1801). Such easily comprehended graphics provide information to thedriver illustrating the particular concerns giving rise to the computeddriver warning index.

Also indicated in FIG. 18 are universally understood graphic symbols(1803) communicating to the driver decisions made in the above describedsystems and methods resulting in prohibiting the use of handheldcellular telephones or wireless devices, hands-free cellular telephoneor wireless devices or any device being used for texting. Here again theparticular decision is displayed in an easily comprehended graphicformat explaining to the driver the results of the above calculations.

Although the embodiments above have been described in considerabledetail, numerous variations and modifications will become apparent tothose skilled in the art once the above disclosure is fully appreciated.For example, embodiments with more or fewer variables to be analyzed asdescribed above are possible. Variations of the artificial intelligenceexpert system analysis may be used. Embodiments for the device controlunit as described above, for example, in FIGS. 2 and 3 may be integratedin various degrees with other motor vehicle telematics or system controlprocessors and sensor systems. In some embodiments the device controlunit of FIGS. 2 and 3 may include only a subset of the capabilitiesindicated in FIG. 3. In some embodiments, the device control unit ofFIGS. 2 and 3 may include additional capabilities not shown herein.While the above disclosure is based on a standard automobile vehicleenvironment, the same teachings set forth herein may be applied to othervehicles such as trucks, buses, military vehicles, emergency vehiclessuch as fire trucks and ambulance and the like. It is intended that thefollowing claims be interpreted to embrace all such variations andmodifications.

The invention claimed is:
 1. An artificial intelligence motor vehicledanger driving warning and control method comprising expert systemmultidimensional analysis of interrelated sensor parameters derived frommonitoring a moving motor vehicle driver and motor vehicle drivingconditions with an electronic, specifically programmed, communicationcomputer system comprising: a step of artificial intelligence decisionmaking utilizing said electronic, specifically programmed, communicationcomputer system; a step of integration of said artificial intelligencemotor vehicle danger driving warning and control method with motorvehicle information processing and display system operations; a step ofmonitoring said driver with a vehicle camera sensor and image analysissoftware wherein said vehicle camera sensor is positioned in front ofsaid driver and provides input to said electronic, specificallyprogrammed, communication computer system; a step of monitoring with alane tracking sensor said motor vehicle lane violations or swerving orexcessive lane changing wherein said lane tracking sensor provides inputto said electronic, specifically programmed, communication computersystem; a step of communicating with at least one external informationsource including at least one navigation and/or at least one roadwaycondition information source with at least one vehicle electroniccommunication transceiver; a step of expert system analysis of one ormore multidimensional matrix combinations of two or more interrelatedmultidimensional sensor parameters with expert defined individualparameter ranges with varying degrees of danger for said ranges; a stepof an expert defining propositional logic inference rules definingmultiple range dependent conditional relationships between said two ormore interrelated multidimensional sensor parameters wherein saidconditional relationships may result in combined degrees of danger thatare more or less than degrees of danger for individual sensorparameters; and a step of providing an expert system derived integratedcomposite degree of danger warning index based on multiple selectedvehicle sensor inputs, including said vehicle camera sensor inputsderived from monitoring said vehicle driver and/or passengers, and saidvehicle lane tracking sensor inputs derived from monitoring said vehiclelane violations, swerving, or excessive lane changing.
 2. The method ofclaim 1 further comprising a step of expert system multidimensionalanalysis of three or more interrelated sensor parameters including atleast motor vehicle driver parameters and motor vehicle drivingcondition parameters.
 3. The method of claim 2 further comprising a stepof including in said three or more interrelated sensor parameters one ormore of traffic monitoring system parameters, weather reporting systemparameters, road condition monitoring system parameters, police or lawenforcement system parameters, emergency alert system parameters and/orpedestrian traffic monitoring system parameters.
 4. The method of claim2 further comprising a step of including in said three or moreinterrelated sensor parameters one or more of tire monitoringparameters, braking monitoring parameters, oil monitoring parameters,temperature monitoring parameters, maintenance monitoring parameters,lighting monitoring parameters, near field communication sensormonitoring parameters, directional microphone array monitoringparameters, RF antenna monitoring parameters, medical sensor monitoringparameters, and/or seat belt sensor monitoring parameters.
 5. The methodof claim 2 further comprising a step of including in said three or moreinterrelated sensor parameters one or more of camera sensor parameters,proximity sensor parameters, road tracking sensor parameters, erraticdriving behavior sensor parameters, location sensor parameters, weathersensor parameters and/or other sensor parameters used to monitor motorvehicle status and driving conditions.
 6. The method of claim 2 furthercomprising a step of including in said three or more interrelated sensorparameters one or more of parameters derived from detecting dangerousdriving conditions including situations wherein the motor vehicle isfollowing vehicles in front too closely, vehicles behind the motorvehicle are following too closely, or motor vehicles being present in a“blind spot” on said motor vehicle driver or passenger side of a movingmotor vehicle using one or more cameras, radar, and/or lidar sensorsmounted on the motor vehicle.
 7. The method of claim 2 furthercomprising a step of including in said three or more interrelated sensorparameters measurements of acceleration of the motor vehicle using anaccelerometer including dangerous or abnormal acceleration.
 8. Themethod of claim 2 further comprising a step of receiving GPS (GlobalPositioning System) signals for tracking changes in location andmovements of the motor vehicle.
 9. The method of claim 2 furthercomprising a step of fuzzy logic expert system multidimensional analysisof said three or more interrelated sensor parameters with overlappingparameter ranges including at least motor vehicle driver parameters andmotor vehicle driving condition parameters.
 10. The method of claim 2further comprising a step of driver eye monitoring with said vehiclesensor camera and image analysis software to verify attention of saiddriver to driving said vehicle.
 11. The method of claim 2 furthercomprising a step of vehicle sensor camera and image analysis softwaredetecting driver drowsiness or sleeping based on opening or closing ofthe driver's eyes and/or position of the driver in his or her seat. 12.The method of claim 2 further comprising a step of vehicle sensor cameraand image analysis software monitoring motions of said driver.
 13. Themethod of claim 2 further comprising a step of the vehicle camera sensorand image analysis software capturing, registering and storing in asystem memory images of one or more authorized principle drivers. 14.The method of claim 13 further comprising a step of said vehicle camerasensor and image analysis software comparing an image of a driveroperating the vehicle to stored authorized principle driver images andtransmitting an alert upon detection of an unauthorized driver.
 15. Themethod of claim 14 further comprising a step of transmitting said alertto an authorized individual or owner of said vehicle with an image ofthe driver permitting said authorized individual or owner to approve useof the vehicle even though the driver is not registered as an approvedprinciple driver.
 16. The method of claim 15 further comprising a stepof maintaining history files of motor vehicle drivers with descriptionsof their driving habits including dangerous driving tendencies andfurther wherein said history files are used in artificial intelligenceexpert system derivation of said degree of danger warning indices. 17.The method of claim 2 further comprising a step of vehicle camera sensorand image analysis software detecting use of a telecommunication deviceor cellular telephone by said driver while driving.
 18. The method ofclaim 17 further comprising a step of monitoring driver cell phone usagefor voice or text messaging.
 19. The method of claim 2 furthercomprising a step of communicating vehicle information and/or artificialintelligence expert system analysis results to RFID tag readers locatedalong highways, tollways, or roadways being traveled by said motorvehicle.
 20. The method of claim 2 further comprising a step of vehiclenavigation, vehicle location marking, and verification that the motorvehicle is traveling on established highways or roadways usingelectronic highway or roadway maps.